-- Logs begin at Wed 2022-08-03 18:00:01 UTC, end at Wed 2022-08-03 19:18:12 UTC. -- Aug 03 18:36:09.610714 dv-cinder-851640-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 03 18:36:09.618145 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 03 18:36:09.620143 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 3 18:36:09 2022] *** Aug 03 18:36:09.620143 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 03 18:36:09.620143 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 03 18:36:09.620143 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: nodename: dv-cinder-851640-2-iscsi Aug 03 18:36:09.620143 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: machine: x86_64 Aug 03 18:36:09.620143 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: clock source: unix Aug 03 18:36:09.620143 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: pcre jit disabled Aug 03 18:36:09.620597 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: detected number of CPU cores: 4 Aug 03 18:36:09.620597 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: current working directory: / Aug 03 18:36:09.620597 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: detected binary path: /usr/bin/uwsgi-core Aug 03 18:36:09.620597 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: your processes number limit is 47680 Aug 03 18:36:09.620597 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: your memory page size is 4096 bytes Aug 03 18:36:09.620597 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: detected max file descriptor number: 2048 Aug 03 18:36:09.620597 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: lock engine: pthread robust mutexes Aug 03 18:36:09.621701 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: thunder lock: enabled Aug 03 18:36:09.621701 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 03 18:36:09.621869 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 03 18:36:09.661863 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: Python main interpreter initialized at 0x55ff061169c0 Aug 03 18:36:09.661863 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: python threads support enabled Aug 03 18:36:09.661863 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: your server socket listen backlog is limited to 100 connections Aug 03 18:36:09.661863 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: your mercy for graceful operations on workers is 90 seconds Aug 03 18:36:09.662261 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: mapped 403077 bytes (393 KB) for 2 cores Aug 03 18:36:09.662261 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: *** Operational MODE: preforking *** Aug 03 18:36:09.662535 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: *** uWSGI is running in multiple interpreter mode *** Aug 03 18:36:09.662535 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: spawned uWSGI master process (pid: 91882) Aug 03 18:36:09.662744 dv-cinder-851640-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 03 18:36:09.663675 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: spawned uWSGI worker 1 (pid: 91894, cores: 1) Aug 03 18:36:09.663675 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: spawned uWSGI worker 2 (pid: 91895, cores: 1) Aug 03 18:36:09.663825 dv-cinder-851640-2-iscsi devstack@keystone.service[91882]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 03 18:36:15.248218 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 18:36:15.248218 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: from cryptography.utils import int_from_bytes Aug 03 18:36:15.249069 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 18:36:15.249069 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: from cryptography.utils import int_from_bytes Aug 03 18:36:15.270456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 18:36:15.270456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: from cryptography.utils import int_from_bytes Aug 03 18:36:15.271056 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 18:36:15.271056 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: from cryptography.utils import int_from_bytes Aug 03 18:36:15.300266 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 03 18:36:15.300493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 03 18:36:15.300690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] command line args: [] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 03 18:36:15.300963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 03 18:36:15.301145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ================================================================================ {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 03 18:36:15.301401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] admin_token = **** {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.301660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] config_dir = [] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.301835 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.302020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] config_source = [] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.302202 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] debug = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.302403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.302897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.302993 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.303193 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.303444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.303680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] list_limit = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.303863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] log_config_append = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.304043 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.304222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] log_dir = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.304430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] log_file = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.304670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.304921 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.305197 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.305533 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.305763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.305946 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.306204 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.306446 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.306685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.306923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.307162 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.307398 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.307649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.307898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.308177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.308698 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.69/identity {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.308883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] publish_errors = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.309118 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.309351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.309558 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.310329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.310567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.310810 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] standard_threads = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.311044 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.311288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.311466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.311644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] use_journal = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.311820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] use_json = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.311996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] use_stderr = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.312173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] use_syslog = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.312447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.312705 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.312945 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.313205 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.313448 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.313698 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.313939 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.314188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.314436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.314684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.314933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.315186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.315424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.315674 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.315913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.316162 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.316437 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.316678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.316934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.317191 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.317439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.317678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.317932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.318202 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.318457 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.318541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 03 18:36:15.318708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.318792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 03 18:36:15.318905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] command line args: [] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 03 18:36:15.318980 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.319162 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.319256 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 03 18:36:15.319335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ================================================================================ {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 03 18:36:15.319405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.319587 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] admin_token = **** {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.319675 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.319834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] config_dir = [] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.319911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.320022 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.320101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.320205 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] config_source = [] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.320370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.320453 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] debug = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.320556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.320635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.320805 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.320892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.321009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.321085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.321259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.321336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.321509 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.321594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.321740 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] list_limit = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.321817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.321919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] log_config_append = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.322001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.322097 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.322202 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.322286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] log_dir = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.322438 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.322515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] log_file = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.322671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.322746 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.322902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.322979 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.323134 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.323212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.323375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.323453 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.323535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.323628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.323705 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.323863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.323940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.324104 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.324183 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.324367 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.324459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.324610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.324695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.324849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.324928 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.325079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.325172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.325345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.325457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.325645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.325731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.325809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.325885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.326042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.326282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.326371 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.69/identity {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.326515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] publish_errors = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.326591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.326749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.326835 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.326978 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.327057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.327155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.327238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.327402 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.327567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.327636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.327808 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.327884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] standard_threads = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.328074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.328150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.328335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.328416 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.328517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.328594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.328693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] use_journal = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.328806 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.328891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] use_json = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.329029 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] use_stderr = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.329111 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.329203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] use_syslog = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.329306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.329435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 18:36:15.329551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.329686 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.329803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.329924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.330054 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.330181 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.330328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.330421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.330531 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.330667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.330744 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.330922 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.331061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.331136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.331251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.331370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.331448 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.331615 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.331702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.331860 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.331938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.332107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.332190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.332367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.332461 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.332615 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.332719 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.332852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.332954 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.333097 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.333206 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.333343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.333448 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.333581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.333692 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.333822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.333941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.334071 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.334145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.334310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.334415 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.334546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.334626 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.334784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.334864 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.335021 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.335102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.335343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.335499 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.335577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.335735 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.335813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.335964 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.336051 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.336195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.336280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.336454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.336543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.336702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.336793 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.336942 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.337019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.337168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.337245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.337399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.337484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.337642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.337727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.337887 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.337965 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] auth.external = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.338124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.338208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.338367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.338444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.338599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.338676 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.338838 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.338920 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] auth.password = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.339074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.339152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] auth.token = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.339308 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.339392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.339539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.339625 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.339770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.339854 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.340000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.340079 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.340235 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.340341 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.340507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.340587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.340766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.340842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.341015 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.341121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] credential.caching = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.341247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.341375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.341499 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.341716 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.341797 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.341881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.341954 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.342094 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.342179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.342340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.342419 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.342578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.342658 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.342891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.342981 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.343079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.343153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.343318 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.343400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.343550 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.343635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.343799 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.343878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.344037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.344126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.344275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.344404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.344529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.344621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.344788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.344879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.345026 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.345105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] federation.caching = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.345263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.345341 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.345503 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.345588 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.345741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.345819 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.345973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.346050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.346223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.346313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.346466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.346545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.346712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.346788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.346950 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.347037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.347185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.347270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.347422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.347501 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.347668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.347746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.347906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity.caching = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.347996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.348145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.348223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.348417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.348500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.348657 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.348747 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.348896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.348974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.349135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.349215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.349366 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.349452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.349600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.349678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.349848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.349940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.350093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.350184 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.350329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.350410 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.350565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.350642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.350801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.350887 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.351038 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.351116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.351276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.351355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.351512 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.351598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.351743 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.351823 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.351974 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.352052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.352206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.352311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.352466 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.352545 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.352698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.352776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.352939 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.353034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.353176 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.353255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.353407 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.353484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.353646 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.353732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.353888 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.354008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.354130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.354209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.354372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.354458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.354605 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.354683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] auth.external = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.354834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.354911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.355091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.355177 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.355330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.355410 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.355573 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] auth.password = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.355651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.355804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] auth.token = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.355892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.356043 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.356123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.356278 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.357079 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.357079 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.357319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.357394 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.357455 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.357515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.357575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.357745 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.357838 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.357984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.358065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.358225 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.358304 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] credential.caching = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.358678 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.358774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.358774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.358896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.358975 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.359048 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.359158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.359240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.359408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.359488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.359642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.359719 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.359884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.359963 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.360122 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.360201 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.360384 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.360468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.360621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.360702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.360864 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.360944 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.361125 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.361206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.361360 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.361441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.361608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.361687 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.361856 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.361942 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.362102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] federation.caching = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.362181 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.362336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.362415 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.362569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.362648 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.362808 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.362890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.363037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.363115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.363274 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.363351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.363508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.363588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.363757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.363836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.363996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.364074 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.364242 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.364335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.364505 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.364640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.364756 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.364896 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.364990 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity.caching = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.365131 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.365211 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.365382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.365461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.365622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.365709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.365863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.365948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.366101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.366181 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.366340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.366426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.366589 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.366668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.366836 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.366919 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.367075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.367161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.367310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.367389 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.367543 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.367622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.367775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.367862 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.368012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.368090 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.368249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.368380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.368515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.368610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.368764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.368846 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.369009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.369088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.369243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.369324 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.369482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.369561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.369725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] resource.caching = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.369810 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.369954 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.370034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.370195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.370275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.370429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.370508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.370667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.370747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.370908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.370987 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.371143 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.371220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.371388 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.371516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.371621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.371702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.371861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.371940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.372757 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] role.caching = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.372893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.373051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.373135 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] role.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.373303 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.373383 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.373552 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.373659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.373787 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.373917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.374029 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.374171 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.374258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.374424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.374504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.374680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.374765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.374932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.375020 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.375171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.375250 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.375429 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.375506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.375667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.375753 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.375917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.375997 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.376168 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.376246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.376420 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.376524 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.376672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.376756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.376920 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.377106 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.377185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.377348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.377428 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.377598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.377677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.377850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.377929 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.378102 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.378265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.378345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.378582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.378661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.378839 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.379006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.379085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.379325 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.379403 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.379564 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.379747 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.379828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.380018 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.380122 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.380559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.380781 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.380874 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.381059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.381138 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.381300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.381486 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.381565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.381764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.381845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.382004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] token.caching = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.382202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.382282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.382478 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.382563 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.382717 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.382924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.383004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.383209 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.383286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.383446 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.383672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.383752 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.383923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.384051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.384171 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.384435 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.384518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.384685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.384847 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.384927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.385155 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.385246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.385404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.385625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.385704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.385868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.385994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.386165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.386339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.386419 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.connection = **** {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.386661 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.386749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.386909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.387040 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.387149 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.387393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.387473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.387754 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.387836 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.388006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.388231 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.388371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.388493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.388620 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.388737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.388976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.389067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.389225 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.389351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.389462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.389948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.390047 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.390047 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.390356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.390436 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.390509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.390627 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.390707 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.390870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.391001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.391080 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91895) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 03 18:36:15.391360 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.391724 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.392095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] resource.caching = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.392515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.392898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.393265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.393648 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.394033 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.394397 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.394775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.395141 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.395508 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.395869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] role.caching = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.396249 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] role.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.396633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.397017 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.397394 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.397773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.398152 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.398529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.398905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.399282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.399661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.400016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.400419 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.400798 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.401175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.401552 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.402183 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.402549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.402931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.403419 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.403783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.404229 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.404691 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.405152 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.405599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.406035 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.406450 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.406858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.407268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.407716 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.408159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.408631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.409070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.409515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.409963 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.410372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] token.caching = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.410628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.410885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.411132 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.411394 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.411647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.411890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.412139 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.412416 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.412671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.412911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.413153 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.413390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.413638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.413889 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.414123 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.414366 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.414671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.414941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.connection = **** {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.415192 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.415442 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.415684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.415934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.416174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.416442 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.416683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.416924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.417172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.417410 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.417657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.417894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.418142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.418529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.419552 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.419886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.420211 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.420657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 18:36:15.420965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91894) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 03 18:36:15.421800 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.422036 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.422263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.430844 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.447956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.448186 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.448442 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.458047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.461827 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.462064 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.481116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.481930 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.518001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.519688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.569397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.569635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.570413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.570653 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.635565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.635798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.636008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.636101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.636233 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.636357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.636477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.636562 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.636692 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.636776 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.636908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.637126 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.660386 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.660529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.660627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.660766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.660872 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.661001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.661088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.661229 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.661315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.661453 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.661540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.661670 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.661755 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.661887 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.673569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.673725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.686434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.686651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.686762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.686887 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.687058 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.687199 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.687342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.687470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.687555 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.687642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.687720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.687794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.687868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91895) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.687942 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91894) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 18:36:15.705152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.705349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.705439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 18:36:15.705561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 18:36:15.705683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.705887 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 18:36:15.705988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.706067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 18:36:15.706140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.706336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.706421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.706594 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.706680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.706851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.706933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.707103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.717190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.717431 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.717526 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.717706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.717794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.717971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.718067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.718249 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.725043 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.725338 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.729845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.730125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.730214 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.730408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.730491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.730687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.730771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.730963 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.731049 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.731219 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.731301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.732133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.732266 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.732437 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.732548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.732709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.732894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.733017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.733170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.733286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.733481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.733750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.756709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.758216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.758500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.760156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.761776 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.762058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.763344 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.763624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.763904 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.766823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.767098 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.767376 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.770774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.771018 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.773974 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.774255 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.775156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.775446 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.775728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.776023 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.776310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.776614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.776880 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.777145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.779052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.779365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.779647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.779925 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.780208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.780517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.780795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.781060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.793763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.794181 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.797917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.798194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.798725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.801896 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.802159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.802289 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.805326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.805621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.806621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.809869 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.810221 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.813047 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.813352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.813483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.813649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.813937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.814226 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.814513 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.816688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.816986 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.817271 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.817557 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.817832 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.818109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.830685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.830972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.831353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.831693 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.835067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.835351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.835632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.835904 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.838638 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.840201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.840505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.843477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.843859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.844144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.844499 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.844851 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.845051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.845204 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.845338 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.848151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.848462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.848742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.849025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.849313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.857272 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.857625 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.858154 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.858440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.858725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.859016 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.859302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.861511 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.862072 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.862349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.862624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.862905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.863184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.875362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.878648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.878990 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.881900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.882248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.884496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.885220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.886005 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.886289 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.886582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.887825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.889454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.889750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.890034 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.893217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.894816 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.896537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.897784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.898077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.898204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.898348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.898630 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.901073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.901370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.901654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.901928 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.907325 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.907623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.907909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.910541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.910824 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.911103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.914853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91895) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.915138 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.915427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.915708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.915984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.916259 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.916559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.916838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.917229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.917512 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.917786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.917984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91894) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 18:36:15.918067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.918268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.918350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.918547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.918631 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91895) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.918943 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.919768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.920005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.920278 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.920579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.921035 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.921244 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.921517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.921785 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.922062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.922338 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91894) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 18:36:15.952683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ff061169c0 pid: 91895 (default app) Aug 03 18:36:15.956145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c1516-ca38-4596-878f-df302e7c0ba1 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:15.956240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ff061169c0 pid: 91894 (default app) Aug 03 18:36:15.956462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c1516-ca38-4596-878f-df302e7c0ba1 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:15.956742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c1516-ca38-4596-878f-df302e7c0ba1 None None] PATH_INFO: `/v3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:15.961039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1/1] 10.222.0.69 () {50 vars in 791 bytes} [Wed Aug 3 18:36:15 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:23.692650 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75bb4a-80b0-44bb-a78d-fe43440c56e4 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:23.693232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75bb4a-80b0-44bb-a78d-fe43440c56e4 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:23.693232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75bb4a-80b0-44bb-a78d-fe43440c56e4 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:23.697606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1/2] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:36:23.703050 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a415597-1380-4bed-8003-b280415508f7 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:23.703488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a415597-1380-4bed-8003-b280415508f7 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:23.703603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a415597-1380-4bed-8003-b280415508f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:23.775245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: INFO dbcounter [None req-8a415597-1380-4bed-8003-b280415508f7 None None] Registered counter for database keystone Aug 03 18:36:23.794926 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG oslo_db.sqlalchemy.engines [None req-8a415597-1380-4bed-8003-b280415508f7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91895) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 03 18:36:23.801251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writer thread running {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 03 18:36:24.007779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG passlib.handlers.bcrypt [None req-8a415597-1380-4bed-8003-b280415508f7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91895) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 03 18:36:24.008187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG passlib.handlers.bcrypt [None req-8a415597-1380-4bed-8003-b280415508f7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91895) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 03 18:36:24.064020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-8a415597-1380-4bed-8003-b280415508f7 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:24.065666 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8a415597-1380-4bed-8003-b280415508f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:24.202797 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 2/3] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 501 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 03 18:36:24.354789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75bb4a-80b0-44bb-a78d-fe43440c56e4 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:24.433148 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: INFO dbcounter [None req-8b75bb4a-80b0-44bb-a78d-fe43440c56e4 None None] Registered counter for database keystone Aug 03 18:36:24.455963 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG oslo_db.sqlalchemy.engines [None req-8b75bb4a-80b0-44bb-a78d-fe43440c56e4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91894) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 03 18:36:24.462262 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writer thread running {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 03 18:36:24.585041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75bb4a-80b0-44bb-a78d-fe43440c56e4 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:24.589319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe42b8e-2e3d-4952-bd30-2ef34b597121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:24.589952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe42b8e-2e3d-4952-bd30-2ef34b597121 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:24.590259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe42b8e-2e3d-4952-bd30-2ef34b597121 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:24.590651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe42b8e-2e3d-4952-bd30-2ef34b597121 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:24.598926 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe42b8e-2e3d-4952-bd30-2ef34b597121 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:24.653879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cbe42b8e-2e3d-4952-bd30-2ef34b597121 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 03 18:36:24.654135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe42b8e-2e3d-4952-bd30-2ef34b597121 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:24.664486 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 2/4] 10.222.0.69 () {58 vars in 1324 bytes} [Wed Aug 3 18:36:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 18:36:24.671304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a415597-1380-4bed-8003-b280415508f7 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:24.686001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a415597-1380-4bed-8003-b280415508f7 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:24.690453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d4178-65b4-4eb2-8633-3857380554f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:24.691567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d4178-65b4-4eb2-8633-3857380554f0 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:24.691843 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d4178-65b4-4eb2-8633-3857380554f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:24.692109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d4178-65b4-4eb2-8633-3857380554f0 admin admin] PATH_INFO: `/v3/projects/649429edefe44474b26fed9c93d5ec67` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:24.693640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d4178-65b4-4eb2-8633-3857380554f0 admin admin] RBAC: Authorizing `identity:get_project(project_id=649429edefe44474b26fed9c93d5ec67)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:24.748920 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: WARNING keystone.common.rbac_enforcer.enforcer [None req-af2d4178-65b4-4eb2-8633-3857380554f0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 03 18:36:24.749506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d4178-65b4-4eb2-8633-3857380554f0 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:24.753947 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 3/5] 10.222.0.69 () {56 vars in 1207 bytes} [Wed Aug 3 18:36:24 2022] GET /identity/v3/projects/649429edefe44474b26fed9c93d5ec67 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:26.793085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166284a-67ae-42bd-8690-3b2247fe9088 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:26.793637 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166284a-67ae-42bd-8690-3b2247fe9088 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:26.793637 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166284a-67ae-42bd-8690-3b2247fe9088 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:26.798580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 3/6] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:26.804358 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87b2ec-a6fa-4819-b5fe-ad5d44368a2f None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:26.804786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87b2ec-a6fa-4819-b5fe-ad5d44368a2f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:26.804920 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87b2ec-a6fa-4819-b5fe-ad5d44368a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:26.825445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-9a87b2ec-a6fa-4819-b5fe-ad5d44368a2f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:26.826973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-9a87b2ec-a6fa-4819-b5fe-ad5d44368a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:26.834920 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 4/7] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 03 18:36:26.857276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166284a-67ae-42bd-8690-3b2247fe9088 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:26.869241 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166284a-67ae-42bd-8690-3b2247fe9088 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:26.874113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083e878-7fb1-4a6c-bebb-b54efd8e13f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:26.874717 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083e878-7fb1-4a6c-bebb-b54efd8e13f6 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:26.874999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083e878-7fb1-4a6c-bebb-b54efd8e13f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:26.875274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083e878-7fb1-4a6c-bebb-b54efd8e13f6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:26.883172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083e878-7fb1-4a6c-bebb-b54efd8e13f6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:26.884968 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083e878-7fb1-4a6c-bebb-b54efd8e13f6 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:26.894060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 4/8] 10.222.0.69 () {58 vars in 1324 bytes} [Wed Aug 3 18:36:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 18:36:26.899679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87b2ec-a6fa-4819-b5fe-ad5d44368a2f None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:26.909261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87b2ec-a6fa-4819-b5fe-ad5d44368a2f None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:26.913668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0955b-2ff5-4974-ae1b-e80d1d39dd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:26.914306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0955b-2ff5-4974-ae1b-e80d1d39dd84 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:26.914590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0955b-2ff5-4974-ae1b-e80d1d39dd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:26.914866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0955b-2ff5-4974-ae1b-e80d1d39dd84 admin admin] PATH_INFO: `/v3/users/b499f31220244d41877273ff02e31ae4` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:26.916244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0955b-2ff5-4974-ae1b-e80d1d39dd84 admin admin] RBAC: Authorizing `identity:get_user(user_id=b499f31220244d41877273ff02e31ae4)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:26.918104 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0955b-2ff5-4974-ae1b-e80d1d39dd84 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:26.920362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 5/9] 10.222.0.69 () {56 vars in 1198 bytes} [Wed Aug 3 18:36:26 2022] GET /identity/v3/users/b499f31220244d41877273ff02e31ae4 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:28.975040 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8ca03-8402-4f24-80f3-cece91b5e2e7 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:28.975540 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8ca03-8402-4f24-80f3-cece91b5e2e7 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:28.975637 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8ca03-8402-4f24-80f3-cece91b5e2e7 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:28.976973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 5/10] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:28.982887 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953b9cb-0a00-4d73-b96c-1aeb0a8a4550 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:28.983331 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953b9cb-0a00-4d73-b96c-1aeb0a8a4550 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:28.983511 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953b9cb-0a00-4d73-b96c-1aeb0a8a4550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:29.003294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-5953b9cb-0a00-4d73-b96c-1aeb0a8a4550 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:29.004509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5953b9cb-0a00-4d73-b96c-1aeb0a8a4550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:29.029305 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297e706-b934-4e28-b442-38483fad52e4 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:29.029731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297e706-b934-4e28-b442-38483fad52e4 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:29.030008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297e706-b934-4e28-b442-38483fad52e4 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:29.031345 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 6/11] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:36:29.036765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c3641-e190-4d34-a7eb-27af6af13de9 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:29.037049 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c3641-e190-4d34-a7eb-27af6af13de9 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:29.037325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c3641-e190-4d34-a7eb-27af6af13de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:29.081264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG passlib.handlers.bcrypt [None req-ad3c3641-e190-4d34-a7eb-27af6af13de9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91894) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 03 18:36:29.081686 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG passlib.handlers.bcrypt [None req-ad3c3641-e190-4d34-a7eb-27af6af13de9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91894) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 03 18:36:29.110489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 6/12] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:29.128153 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ad3c3641-e190-4d34-a7eb-27af6af13de9 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:29.129810 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ad3c3641-e190-4d34-a7eb-27af6af13de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:29.133391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953b9cb-0a00-4d73-b96c-1aeb0a8a4550 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:29.158543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:29.233583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 7/13] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:36:29.240617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953b9cb-0a00-4d73-b96c-1aeb0a8a4550 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:29.256069 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c3641-e190-4d34-a7eb-27af6af13de9 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:29.275892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84683127-6d3b-480f-bbe8-8975e1d52dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:29.276579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84683127-6d3b-480f-bbe8-8975e1d52dd6 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:29.276861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84683127-6d3b-480f-bbe8-8975e1d52dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:29.277137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84683127-6d3b-480f-bbe8-8975e1d52dd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:29.288982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84683127-6d3b-480f-bbe8-8975e1d52dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-84683127-6d3b-480f-bbe8-8975e1d52dd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:29.298211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:29.299234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:29.300057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:29.300057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:29.300057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:29.300057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:29.300057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:29.300057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:29.300057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:29.300057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:29.300057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:29.300057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:29.300057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:36:29.300057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:36:29.300057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 7/14] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:36:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:36:29.305211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84683127-6d3b-480f-bbe8-8975e1d52dd6 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:29.362546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c3641-e190-4d34-a7eb-27af6af13de9 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:29.365296 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:29.390780 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:29.404898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba758b-1550-43a2-be1b-9cede883c8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:29.405835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba758b-1550-43a2-be1b-9cede883c8ad None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:29.406124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba758b-1550-43a2-be1b-9cede883c8ad None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:29.406399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba758b-1550-43a2-be1b-9cede883c8ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:29.411590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84683127-6d3b-480f-bbe8-8975e1d52dd6 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:29.446662 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb973bad-68eb-4658-8c31-4feaf51db9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:29.447285 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb973bad-68eb-4658-8c31-4feaf51db9c6 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:29.447571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb973bad-68eb-4658-8c31-4feaf51db9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:29.447849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb973bad-68eb-4658-8c31-4feaf51db9c6 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:29.448615 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb973bad-68eb-4658-8c31-4feaf51db9c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:29.448926 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb973bad-68eb-4658-8c31-4feaf51db9c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:29.450988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb973bad-68eb-4658-8c31-4feaf51db9c6 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:29.464665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 8/15] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:29.470304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb973bad-68eb-4658-8c31-4feaf51db9c6 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:29.475554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba758b-1550-43a2-be1b-9cede883c8ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:29.477154 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba758b-1550-43a2-be1b-9cede883c8ad None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:29.581324 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 8/16] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:36:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 18:36:29.588226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba758b-1550-43a2-be1b-9cede883c8ad None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:29.591880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:29.677717 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb973bad-68eb-4658-8c31-4feaf51db9c6 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:29.691442 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba758b-1550-43a2-be1b-9cede883c8ad None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:29.711123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f193a-33ce-4578-9fd8-779256b0e2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:29.711772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f193a-33ce-4578-9fd8-779256b0e2ca None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:29.712054 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f193a-33ce-4578-9fd8-779256b0e2ca None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:29.712358 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f193a-33ce-4578-9fd8-779256b0e2ca None admin] PATH_INFO: `/v3/users/b499f31220244d41877273ff02e31ae4` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:29.713522 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f193a-33ce-4578-9fd8-779256b0e2ca None admin] RBAC: Authorizing `identity:get_user(user_id=b499f31220244d41877273ff02e31ae4)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:29.715026 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f193a-33ce-4578-9fd8-779256b0e2ca None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:29.716751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 9/17] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:29 2022] GET /identity/v3/users/b499f31220244d41877273ff02e31ae4 => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:29.722025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f193a-33ce-4578-9fd8-779256b0e2ca None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:29.725041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957f58a-bb6f-401a-8f9e-bf675e9fac52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:29.725913 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957f58a-bb6f-401a-8f9e-bf675e9fac52 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:29.726202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957f58a-bb6f-401a-8f9e-bf675e9fac52 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:29.726478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957f58a-bb6f-401a-8f9e-bf675e9fac52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:29.737318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:29.745536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957f58a-bb6f-401a-8f9e-bf675e9fac52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:29.747733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957f58a-bb6f-401a-8f9e-bf675e9fac52 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:29.765999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-7957f58a-bb6f-401a-8f9e-bf675e9fac52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:36:29.765999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:29.765999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:29.765999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:29.765999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:29.765999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:29.765999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:29.765999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:29.765999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:29.765999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:29.765999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:29.765999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:29.765999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:36:29.765999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:36:29.765999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:29.766895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:29.767712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:36:29.767712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:36:29.767712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:36:29.767712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:36:29.767712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 9/18] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:36:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:36:29.773236 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957f58a-bb6f-401a-8f9e-bf675e9fac52 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:29.827920 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f193a-33ce-4578-9fd8-779256b0e2ca None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:29.831562 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:29.862345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc41081-4091-46f1-83dc-10619ff4737e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:29.862995 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc41081-4091-46f1-83dc-10619ff4737e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:29.863282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc41081-4091-46f1-83dc-10619ff4737e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:29.863560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc41081-4091-46f1-83dc-10619ff4737e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:29.864765 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc41081-4091-46f1-83dc-10619ff4737e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:29.866239 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc41081-4091-46f1-83dc-10619ff4737e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:29.867827 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 10/19] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:36:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:29.874521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc41081-4091-46f1-83dc-10619ff4737e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:29.877067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957f58a-bb6f-401a-8f9e-bf675e9fac52 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:29.912023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b783db4-063b-48a9-bda5-059e2f67f9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:29.912677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b783db4-063b-48a9-bda5-059e2f67f9e5 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:29.912962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b783db4-063b-48a9-bda5-059e2f67f9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:29.913239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b783db4-063b-48a9-bda5-059e2f67f9e5 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:29.913949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b783db4-063b-48a9-bda5-059e2f67f9e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:29.914260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b783db4-063b-48a9-bda5-059e2f67f9e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:29.915725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b783db4-063b-48a9-bda5-059e2f67f9e5 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:29.933915 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 10/20] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:29.960262 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:29.982957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc41081-4091-46f1-83dc-10619ff4737e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:30.020341 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab1ba7-d90f-4582-b23a-b77d2e2efb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:30.020982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab1ba7-d90f-4582-b23a-b77d2e2efb85 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:30.021263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab1ba7-d90f-4582-b23a-b77d2e2efb85 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:30.021539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab1ba7-d90f-4582-b23a-b77d2e2efb85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:30.022348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab1ba7-d90f-4582-b23a-b77d2e2efb85 None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.domain.id=default, user.id=b499f31220244d41877273ff02e31ae4) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:30.022658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab1ba7-d90f-4582-b23a-b77d2e2efb85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:30.024139 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab1ba7-d90f-4582-b23a-b77d2e2efb85 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:30.032601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 11/21] 10.222.0.69 () {56 vars in 1322 bytes} [Wed Aug 3 18:36:29 2022] GET /identity/v3/role_assignments?user.id=b499f31220244d41877273ff02e31ae4&scope.domain.id=default&role.id=19557e941d6f475ba92cf862846f9872 => generated 233 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:32.342551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988df2f-97c6-4142-8cd0-1d196b0b29ce None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:32.343658 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988df2f-97c6-4142-8cd0-1d196b0b29ce None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:32.344164 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988df2f-97c6-4142-8cd0-1d196b0b29ce None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:32.345779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 11/22] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:32.351253 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af43f4c-4dcb-4ff0-89bf-9c2879d1a832 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:32.351658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af43f4c-4dcb-4ff0-89bf-9c2879d1a832 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:32.351807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af43f4c-4dcb-4ff0-89bf-9c2879d1a832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:32.381796 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-9af43f4c-4dcb-4ff0-89bf-9c2879d1a832 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:32.383257 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-9af43f4c-4dcb-4ff0-89bf-9c2879d1a832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:32.477897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 12/23] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:36:32.511966 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988df2f-97c6-4142-8cd0-1d196b0b29ce None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:32.648184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988df2f-97c6-4142-8cd0-1d196b0b29ce None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:32.669096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:32.692218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d400d0-b718-4508-9c93-741e510dfcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:32.692918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d400d0-b718-4508-9c93-741e510dfcfb None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:32.693206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d400d0-b718-4508-9c93-741e510dfcfb None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:32.693541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d400d0-b718-4508-9c93-741e510dfcfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:32.708071 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d400d0-b718-4508-9c93-741e510dfcfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:32.709589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d400d0-b718-4508-9c93-741e510dfcfb None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:32.730327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-96d400d0-b718-4508-9c93-741e510dfcfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:36:32.730327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:32.730327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:32.730327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:32.730327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:32.730327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:32.730327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:32.730327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:32.730327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:32.730327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:32.730327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:32.730327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:32.730327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:36:32.730327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:36:32.730327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:32.731881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:32.732831 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:36:32.732831 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:36:32.732831 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:36:32.732831 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:36:32.733914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 12/24] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:32.738568 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af43f4c-4dcb-4ff0-89bf-9c2879d1a832 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:32.814493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69878052-36e2-4714-af29-0462c338897d None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:32.815121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69878052-36e2-4714-af29-0462c338897d None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:32.815617 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69878052-36e2-4714-af29-0462c338897d None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:32.818523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 13/25] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:32.825067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324579a6-abd8-47ec-bc10-95724799eca9 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:32.825629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324579a6-abd8-47ec-bc10-95724799eca9 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:32.826108 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324579a6-abd8-47ec-bc10-95724799eca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:32.839929 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:32.875108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af43f4c-4dcb-4ff0-89bf-9c2879d1a832 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:32.961105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41d43c-2d0d-41ec-8b4d-be17a648239f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:32.961776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41d43c-2d0d-41ec-8b4d-be17a648239f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:32.962058 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41d43c-2d0d-41ec-8b4d-be17a648239f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:32.962335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41d43c-2d0d-41ec-8b4d-be17a648239f None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:32.963075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41d43c-2d0d-41ec-8b4d-be17a648239f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:32.963382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41d43c-2d0d-41ec-8b4d-be17a648239f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:32.964874 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41d43c-2d0d-41ec-8b4d-be17a648239f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:32.999937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 13/26] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:33.001093 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34885f78-5669-4c6e-9b59-655a029998cd None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:33.001416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34885f78-5669-4c6e-9b59-655a029998cd None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:33.001973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34885f78-5669-4c6e-9b59-655a029998cd None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:33.003416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 14/27] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:33.008433 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34885f78-5669-4c6e-9b59-655a029998cd None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:33.015420 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-324579a6-abd8-47ec-bc10-95724799eca9 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:33.016679 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-324579a6-abd8-47ec-bc10-95724799eca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:33.114354 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 14/28] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:36:33.115967 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77402bc0-79d4-4313-ad04-fe59684f4411 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:33.116441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77402bc0-79d4-4313-ad04-fe59684f4411 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:33.116854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77402bc0-79d4-4313-ad04-fe59684f4411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:33.135988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-77402bc0-79d4-4313-ad04-fe59684f4411 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:33.137579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-77402bc0-79d4-4313-ad04-fe59684f4411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:33.152016 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34885f78-5669-4c6e-9b59-655a029998cd None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:33.197738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81b4af-ab86-42a5-93e7-80a35375c9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:33.198586 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81b4af-ab86-42a5-93e7-80a35375c9b4 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:33.199022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81b4af-ab86-42a5-93e7-80a35375c9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:33.199523 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81b4af-ab86-42a5-93e7-80a35375c9b4 None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:33.201656 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b81b4af-ab86-42a5-93e7-80a35375c9b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:33.203636 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b81b4af-ab86-42a5-93e7-80a35375c9b4 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:33.210688 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:33.217032 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 15/29] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:33.218605 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ce092-2f6e-491b-8c98-c7e0e28d6869 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:33.219806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ce092-2f6e-491b-8c98-c7e0e28d6869 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:33.220237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ce092-2f6e-491b-8c98-c7e0e28d6869 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:33.222158 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 16/30] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:33.222987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ce092-2f6e-491b-8c98-c7e0e28d6869 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:33.256423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:33.337797 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ce092-2f6e-491b-8c98-c7e0e28d6869 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:33.368666 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 15/31] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Aug 3 18:36:33 2022] POST /identity/v3/projects => generated 301 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:36:33.370444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6feb4-18db-4ce0-9fd4-cead9f8b35d6 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:33.370804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6feb4-18db-4ce0-9fd4-cead9f8b35d6 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:33.371849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6feb4-18db-4ce0-9fd4-cead9f8b35d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:33.376383 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ca5d9-18d1-4591-9195-959192a6df08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:33.377758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ca5d9-18d1-4591-9195-959192a6df08 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:33.378218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ca5d9-18d1-4591-9195-959192a6df08 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:33.378715 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ca5d9-18d1-4591-9195-959192a6df08 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:33.380054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ca5d9-18d1-4591-9195-959192a6df08 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:33.381827 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ca5d9-18d1-4591-9195-959192a6df08 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:33.396161 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d8c6feb4-18db-4ce0-9fd4-cead9f8b35d6 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:33.397866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d8c6feb4-18db-4ce0-9fd4-cead9f8b35d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:33.491334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 16/32] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:33.492197 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6feb4-18db-4ce0-9fd4-cead9f8b35d6 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:33.524019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 17/33] 10.222.0.69 () {60 vars in 1131 bytes} [Wed Aug 3 18:36:33 2022] POST /identity/v3/roles => generated 219 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:36:33.576068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ca5d9-18d1-4591-9195-959192a6df08 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:33.669020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6feb4-18db-4ce0-9fd4-cead9f8b35d6 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:33.690346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:33.715158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa67dc-37e9-4832-8a3f-b363bcdbe442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:33.716425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa67dc-37e9-4832-8a3f-b363bcdbe442 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:33.716425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa67dc-37e9-4832-8a3f-b363bcdbe442 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:33.716621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa67dc-37e9-4832-8a3f-b363bcdbe442 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:33.717304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa67dc-37e9-4832-8a3f-b363bcdbe442 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:33.719991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa67dc-37e9-4832-8a3f-b363bcdbe442 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:33.733471 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:33.741229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 17/34] 10.222.0.69 () {60 vars in 1131 bytes} [Wed Aug 3 18:36:33 2022] POST /identity/v3/roles => generated 225 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:36:33.743828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e3a45-ce21-42e4-9862-5db4ccbfcf9f None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:33.744540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e3a45-ce21-42e4-9862-5db4ccbfcf9f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:33.745038 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e3a45-ce21-42e4-9862-5db4ccbfcf9f None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:33.747380 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 18/35] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:33.756231 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab39b6a-6679-4812-acdd-78e372076f5c None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:33.757012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab39b6a-6679-4812-acdd-78e372076f5c None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:33.757516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab39b6a-6679-4812-acdd-78e372076f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:33.759464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ca5d9-18d1-4591-9195-959192a6df08 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:33.784898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-eab39b6a-6679-4812-acdd-78e372076f5c None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:33.785504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-eab39b6a-6679-4812-acdd-78e372076f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:33.822318 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345229c-491a-47fc-bda6-96dbe3e56fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:33.823999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345229c-491a-47fc-bda6-96dbe3e56fbe None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:33.825175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345229c-491a-47fc-bda6-96dbe3e56fbe None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:33.825657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345229c-491a-47fc-bda6-96dbe3e56fbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:33.843702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345229c-491a-47fc-bda6-96dbe3e56fbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-5345229c-491a-47fc-bda6-96dbe3e56fbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:33.853142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:33.854051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:33.854845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:33.854845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:33.854845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:33.854845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:33.854845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:33.854845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:33.854845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:33.854845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:33.854845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:33.854845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:33.854845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:36:33.854845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:36:33.857984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 18/36] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:36:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:33.866087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345229c-491a-47fc-bda6-96dbe3e56fbe None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:33.885487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 19/37] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:33.919502 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab39b6a-6679-4812-acdd-78e372076f5c None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:34.035463 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345229c-491a-47fc-bda6-96dbe3e56fbe None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:34.057236 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab39b6a-6679-4812-acdd-78e372076f5c None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:34.064926 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:34.070529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf289d-4c21-4ffc-a580-b28dfa618112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:34.071156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf289d-4c21-4ffc-a580-b28dfa618112 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:34.071439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf289d-4c21-4ffc-a580-b28dfa618112 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:34.071715 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf289d-4c21-4ffc-a580-b28dfa618112 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:34.072555 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf289d-4c21-4ffc-a580-b28dfa618112 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:34.072861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf289d-4c21-4ffc-a580-b28dfa618112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:34.075047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf289d-4c21-4ffc-a580-b28dfa618112 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:34.086633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 19/38] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:34.087878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed5c11-9181-4197-9515-8609f56473aa None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:34.088167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed5c11-9181-4197-9515-8609f56473aa None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:34.088470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed5c11-9181-4197-9515-8609f56473aa None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:34.089674 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 20/39] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:34.093710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed5c11-9181-4197-9515-8609f56473aa None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:34.097807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44696f19-0070-4ef7-9e6e-841b16d1fca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:34.098446 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44696f19-0070-4ef7-9e6e-841b16d1fca9 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:34.098730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44696f19-0070-4ef7-9e6e-841b16d1fca9 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:34.099009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44696f19-0070-4ef7-9e6e-841b16d1fca9 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:34.099742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44696f19-0070-4ef7-9e6e-841b16d1fca9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:34.101232 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44696f19-0070-4ef7-9e6e-841b16d1fca9 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:34.106737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:34.116167 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 20/40] 10.222.0.69 () {60 vars in 1131 bytes} [Wed Aug 3 18:36:33 2022] POST /identity/v3/roles => generated 223 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:36:34.117413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4bab5-8614-497b-a876-4318301eb9f4 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:34.117700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4bab5-8614-497b-a876-4318301eb9f4 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:34.117980 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4bab5-8614-497b-a876-4318301eb9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:34.138395 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-b9d4bab5-8614-497b-a876-4318301eb9f4 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:34.139689 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b9d4bab5-8614-497b-a876-4318301eb9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:34.216045 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 21/41] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:34.223811 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed5c11-9181-4197-9515-8609f56473aa None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:34.239804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4bab5-8614-497b-a876-4318301eb9f4 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:34.269483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2f80c-8a29-4086-8e1e-65d6e191d0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:34.270412 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2f80c-8a29-4086-8e1e-65d6e191d0a5 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:34.270943 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2f80c-8a29-4086-8e1e-65d6e191d0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:34.271228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2f80c-8a29-4086-8e1e-65d6e191d0a5 None admin] PATH_INFO: `/v3/users/b499f31220244d41877273ff02e31ae4` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:34.273683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2f80c-8a29-4086-8e1e-65d6e191d0a5 None admin] RBAC: Authorizing `identity:get_user(user_id=b499f31220244d41877273ff02e31ae4)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:34.275342 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2f80c-8a29-4086-8e1e-65d6e191d0a5 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:34.277175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 21/42] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:34 2022] GET /identity/v3/users/b499f31220244d41877273ff02e31ae4 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:34.283762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2f80c-8a29-4086-8e1e-65d6e191d0a5 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:34.361314 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:34.368362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4bab5-8614-497b-a876-4318301eb9f4 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:34.377812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:34.408816 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2f80c-8a29-4086-8e1e-65d6e191d0a5 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:34.412385 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c23a0-191f-45ed-8f0b-1d593281d8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:34.413036 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c23a0-191f-45ed-8f0b-1d593281d8a8 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:34.413319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c23a0-191f-45ed-8f0b-1d593281d8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:34.413596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c23a0-191f-45ed-8f0b-1d593281d8a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:34.414766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c23a0-191f-45ed-8f0b-1d593281d8a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:34.416216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c23a0-191f-45ed-8f0b-1d593281d8a8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:34.417830 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 22/43] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:36:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:34.424183 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c23a0-191f-45ed-8f0b-1d593281d8a8 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:34.446232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34535982-bd62-4e2e-ae5f-e6ac1f8273de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:34.446880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34535982-bd62-4e2e-ae5f-e6ac1f8273de None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:34.447162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34535982-bd62-4e2e-ae5f-e6ac1f8273de None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:34.447432 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34535982-bd62-4e2e-ae5f-e6ac1f8273de None admin] PATH_INFO: `/v3/domains/default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:34.448702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34535982-bd62-4e2e-ae5f-e6ac1f8273de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:34.450610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34535982-bd62-4e2e-ae5f-e6ac1f8273de None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:34.452195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 22/44] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:36:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:34.457742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34535982-bd62-4e2e-ae5f-e6ac1f8273de None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:34.524721 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c23a0-191f-45ed-8f0b-1d593281d8a8 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:34.559468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13eff46-df79-4715-a3cb-7361264b68b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:34.560263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13eff46-df79-4715-a3cb-7361264b68b9 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:34.560587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13eff46-df79-4715-a3cb-7361264b68b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:34.560865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13eff46-df79-4715-a3cb-7361264b68b9 None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:34.561318 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34535982-bd62-4e2e-ae5f-e6ac1f8273de None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:34.561556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13eff46-df79-4715-a3cb-7361264b68b9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:34.563022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13eff46-df79-4715-a3cb-7361264b68b9 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:34.590547 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 23/45] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Aug 3 18:36:34 2022] POST /identity/v3/projects => generated 312 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:36:34.596674 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af218df6-f18a-484c-9685-28d9d5c61b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:34.597392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af218df6-f18a-484c-9685-28d9d5c61b07 None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:34.597672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af218df6-f18a-484c-9685-28d9d5c61b07 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:34.597946 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af218df6-f18a-484c-9685-28d9d5c61b07 None admin] PATH_INFO: `/v3/domains/default/users/b499f31220244d41877273ff02e31ae4/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:34.600102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af218df6-f18a-484c-9685-28d9d5c61b07 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b499f31220244d41877273ff02e31ae4, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:34.601631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af218df6-f18a-484c-9685-28d9d5c61b07 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:34.618630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 23/46] 10.222.0.69 () {58 vars in 1361 bytes} [Wed Aug 3 18:36:34 2022] PUT /identity/v3/domains/default/users/b499f31220244d41877273ff02e31ae4/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:36:36.619305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685872ba-9294-4eee-a269-85038eaf204a None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:36.619859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685872ba-9294-4eee-a269-85038eaf204a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:36.619859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685872ba-9294-4eee-a269-85038eaf204a None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:36.621276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 24/47] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:36.626583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de005c-1c82-40a6-84d3-d2d0db0b2094 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:36.626999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de005c-1c82-40a6-84d3-d2d0db0b2094 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:36.627149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de005c-1c82-40a6-84d3-d2d0db0b2094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:36.640937 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:36.646860 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-e6de005c-1c82-40a6-84d3-d2d0db0b2094 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:36.648935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e6de005c-1c82-40a6-84d3-d2d0db0b2094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:36.649125 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab5e7d-3938-4fcf-8b4c-c5e5cda28094 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:36.649419 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab5e7d-3938-4fcf-8b4c-c5e5cda28094 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:36.649701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab5e7d-3938-4fcf-8b4c-c5e5cda28094 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:36.651049 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 25/48] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:36:36.657374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cdc15-ba95-40d7-9d5b-bf42472dc15a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:36.657774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cdc15-ba95-40d7-9d5b-bf42472dc15a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:36.658052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cdc15-ba95-40d7-9d5b-bf42472dc15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:36.676863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-487cdc15-ba95-40d7-9d5b-bf42472dc15a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:36.678073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-487cdc15-ba95-40d7-9d5b-bf42472dc15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:36.690241 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:36.720576 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 24/49] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:36.744373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de005c-1c82-40a6-84d3-d2d0db0b2094 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:36.750547 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 26/50] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:36:36.773704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487cdc15-ba95-40d7-9d5b-bf42472dc15a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:36.845491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de005c-1c82-40a6-84d3-d2d0db0b2094 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:36.873630 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487cdc15-ba95-40d7-9d5b-bf42472dc15a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:36.880005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ef835-3f0f-4624-84c2-b76a65107833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:36.880691 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ef835-3f0f-4624-84c2-b76a65107833 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:36.880979 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ef835-3f0f-4624-84c2-b76a65107833 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:36.881253 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ef835-3f0f-4624-84c2-b76a65107833 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:36.882400 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ef835-3f0f-4624-84c2-b76a65107833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:36.883848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ef835-3f0f-4624-84c2-b76a65107833 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:36.885468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 25/51] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:36:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:36.891971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ef835-3f0f-4624-84c2-b76a65107833 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:36.895903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:36.909864 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54fd7c2-7410-45fc-8e21-05b82657f214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:36.910519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fd7c2-7410-45fc-8e21-05b82657f214 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:36.910800 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fd7c2-7410-45fc-8e21-05b82657f214 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:36.911082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fd7c2-7410-45fc-8e21-05b82657f214 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:36.925004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54fd7c2-7410-45fc-8e21-05b82657f214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-c54fd7c2-7410-45fc-8e21-05b82657f214 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:36.930017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:36.930892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:36.931649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:36.931649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:36.931649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:36.931649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:36.931649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:36.931649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:36.931649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:36.931649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:36.931649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:36.931649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:36.931649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:36:36.931649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:36:36.931649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 27/52] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:36:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:36:36.938136 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54fd7c2-7410-45fc-8e21-05b82657f214 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:36.947162 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:36.993973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ef835-3f0f-4624-84c2-b76a65107833 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:37.030094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a511a3-3758-44a7-b465-c23b31f615d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:37.031004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a511a3-3758-44a7-b465-c23b31f615d1 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:37.031314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a511a3-3758-44a7-b465-c23b31f615d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:37.031565 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a511a3-3758-44a7-b465-c23b31f615d1 None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:37.032254 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a511a3-3758-44a7-b465-c23b31f615d1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:37.033755 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a511a3-3758-44a7-b465-c23b31f615d1 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:37.042418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54fd7c2-7410-45fc-8e21-05b82657f214 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:37.068967 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 26/53] 10.222.0.69 () {60 vars in 1140 bytes} [Wed Aug 3 18:36:36 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:36:37.081062 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44c7bb-9599-42a6-bf34-eb757b5db898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:37.081713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44c7bb-9599-42a6-bf34-eb757b5db898 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:37.081995 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44c7bb-9599-42a6-bf34-eb757b5db898 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:37.082267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44c7bb-9599-42a6-bf34-eb757b5db898 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:37.083067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d44c7bb-9599-42a6-bf34-eb757b5db898 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:37.083376 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d44c7bb-9599-42a6-bf34-eb757b5db898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:37.084949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d44c7bb-9599-42a6-bf34-eb757b5db898 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:37.095151 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 28/54] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:37.101023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a511a3-3758-44a7-b465-c23b31f615d1 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:37.163140 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:37.211190 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a511a3-3758-44a7-b465-c23b31f615d1 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:37.248881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2bdac-c952-4bce-8baf-1e51d22bb13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:37.249551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2bdac-c952-4bce-8baf-1e51d22bb13a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:37.249838 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2bdac-c952-4bce-8baf-1e51d22bb13a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:37.250120 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2bdac-c952-4bce-8baf-1e51d22bb13a None admin] PATH_INFO: `/v3/users/b499f31220244d41877273ff02e31ae4` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:37.251340 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2bdac-c952-4bce-8baf-1e51d22bb13a None admin] RBAC: Authorizing `identity:get_user(user_id=b499f31220244d41877273ff02e31ae4)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:37.252853 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2bdac-c952-4bce-8baf-1e51d22bb13a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:37.254459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 27/55] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:37 2022] GET /identity/v3/users/b499f31220244d41877273ff02e31ae4 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:37.260185 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44c7bb-9599-42a6-bf34-eb757b5db898 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:37.331521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:37.367134 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44c7bb-9599-42a6-bf34-eb757b5db898 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:37.400562 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd618c-e0d2-4ef8-8179-850cebb3ce6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:37.401210 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd618c-e0d2-4ef8-8179-850cebb3ce6c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:37.401495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd618c-e0d2-4ef8-8179-850cebb3ce6c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:37.401774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd618c-e0d2-4ef8-8179-850cebb3ce6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:37.403774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd618c-e0d2-4ef8-8179-850cebb3ce6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:37.405267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd618c-e0d2-4ef8-8179-850cebb3ce6c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:37.406870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 29/56] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:36:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:37.412670 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2bdac-c952-4bce-8baf-1e51d22bb13a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:37.521428 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2bdac-c952-4bce-8baf-1e51d22bb13a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:37.544627 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:37.555690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c70ec-bbe8-4d01-8d38-156ef48af47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:37.556344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c70ec-bbe8-4d01-8d38-156ef48af47d None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:37.556636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c70ec-bbe8-4d01-8d38-156ef48af47d None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:37.556913 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c70ec-bbe8-4d01-8d38-156ef48af47d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:37.557718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c70ec-bbe8-4d01-8d38-156ef48af47d None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.domain.id=default, user.id=b499f31220244d41877273ff02e31ae4) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:37.558028 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c70ec-bbe8-4d01-8d38-156ef48af47d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:37.560187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c70ec-bbe8-4d01-8d38-156ef48af47d None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:37.568544 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 28/57] 10.222.0.69 () {56 vars in 1322 bytes} [Wed Aug 3 18:36:37 2022] GET /identity/v3/role_assignments?user.id=b499f31220244d41877273ff02e31ae4&scope.domain.id=default&role.id=19557e941d6f475ba92cf862846f9872 => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:39.097320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7cec7-28e3-41d3-835b-1528a7ef2795 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:39.097826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7cec7-28e3-41d3-835b-1528a7ef2795 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:39.097914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7cec7-28e3-41d3-835b-1528a7ef2795 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:39.099220 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 30/58] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:39.104534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ec2ad-09d9-4be3-bd26-191951cdfdf2 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:39.104908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ec2ad-09d9-4be3-bd26-191951cdfdf2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:39.105100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ec2ad-09d9-4be3-bd26-191951cdfdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:39.123499 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-1e9ec2ad-09d9-4be3-bd26-191951cdfdf2 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:39.124714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1e9ec2ad-09d9-4be3-bd26-191951cdfdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:39.189715 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 29/59] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:39.213872 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7cec7-28e3-41d3-835b-1528a7ef2795 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:39.316217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7cec7-28e3-41d3-835b-1528a7ef2795 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:39.348712 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e6e9e-5731-4ce2-8825-35c42b299ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:39.349347 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e6e9e-5731-4ce2-8825-35c42b299ec2 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:39.349628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e6e9e-5731-4ce2-8825-35c42b299ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:39.349899 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e6e9e-5731-4ce2-8825-35c42b299ec2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:39.351033 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e6e9e-5731-4ce2-8825-35c42b299ec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:39.352507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e6e9e-5731-4ce2-8825-35c42b299ec2 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:39.354076 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 31/60] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:36:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:39.360319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ec2ad-09d9-4be3-bd26-191951cdfdf2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:39.460922 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ec2ad-09d9-4be3-bd26-191951cdfdf2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:39.477737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:39.495285 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70083d-b1eb-4199-8f3d-0050df176cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:39.497019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70083d-b1eb-4199-8f3d-0050df176cac None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:39.497312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70083d-b1eb-4199-8f3d-0050df176cac None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:39.497591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70083d-b1eb-4199-8f3d-0050df176cac None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:39.498288 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70083d-b1eb-4199-8f3d-0050df176cac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:39.499745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70083d-b1eb-4199-8f3d-0050df176cac None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:39.525458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 30/61] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:36:39 2022] POST /identity/v3/users => generated 275 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:36:42.633450 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddffd8-6f3a-494f-beac-71b6731788fa None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:42.633960 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddffd8-6f3a-494f-beac-71b6731788fa None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:42.634033 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddffd8-6f3a-494f-beac-71b6731788fa None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:42.635660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 32/62] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:42.641179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37cc67-8031-407c-9c5d-69bde9e729da None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:42.641581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37cc67-8031-407c-9c5d-69bde9e729da None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:42.641779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37cc67-8031-407c-9c5d-69bde9e729da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:42.669354 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-3e37cc67-8031-407c-9c5d-69bde9e729da None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:42.670569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-3e37cc67-8031-407c-9c5d-69bde9e729da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:42.755606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 31/63] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:42.800945 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddffd8-6f3a-494f-beac-71b6731788fa None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:42.804918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:42.869423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72b81d-dc11-4ff5-a421-2f715cc9e964 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:42.870049 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72b81d-dc11-4ff5-a421-2f715cc9e964 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:42.870550 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72b81d-dc11-4ff5-a421-2f715cc9e964 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:42.872464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 32/64] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:42.877669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2ec9b-2572-4deb-9936-6b04ce4798c1 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:42.878204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2ec9b-2572-4deb-9936-6b04ce4798c1 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:42.878685 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2ec9b-2572-4deb-9936-6b04ce4798c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:42.903230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-e6f2ec9b-2572-4deb-9936-6b04ce4798c1 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:42.904535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e6f2ec9b-2572-4deb-9936-6b04ce4798c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:42.955474 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddffd8-6f3a-494f-beac-71b6731788fa None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:42.995412 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 33/65] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:36:42.997859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57427b-c1af-438f-9141-fb7d85f8a71a None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:42.998259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57427b-c1af-438f-9141-fb7d85f8a71a None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:42.998766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57427b-c1af-438f-9141-fb7d85f8a71a None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:42.999352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36331e56-502b-4856-8593-af0fdab2ae5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:43.000163 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36331e56-502b-4856-8593-af0fdab2ae5c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:43.000633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36331e56-502b-4856-8593-af0fdab2ae5c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:43.000748 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 34/66] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:43.001200 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36331e56-502b-4856-8593-af0fdab2ae5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:43.008105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b1276-5779-4c66-b0e6-9f09920ba985 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:43.008437 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b1276-5779-4c66-b0e6-9f09920ba985 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:43.008724 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b1276-5779-4c66-b0e6-9f09920ba985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:43.023979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36331e56-502b-4856-8593-af0fdab2ae5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-36331e56-502b-4856-8593-af0fdab2ae5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:43.029660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:43.030530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:43.031411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:43.031411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:43.031411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:43.031411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:43.031411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:43.031411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:43.031411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:43.031411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:43.031411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:43.031411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:43.031411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:36:43.031411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:36:43.034046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 33/67] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:36:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:43.035281 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36331e56-502b-4856-8593-af0fdab2ae5c None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:43.041011 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-cf2b1276-5779-4c66-b0e6-9f09920ba985 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:43.042466 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-cf2b1276-5779-4c66-b0e6-9f09920ba985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:43.080119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:43.116743 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:43.141408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 35/68] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:43.142832 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b1276-5779-4c66-b0e6-9f09920ba985 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:43.183286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36331e56-502b-4856-8593-af0fdab2ae5c None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:43.223283 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c5114-3d17-41c4-92f1-dacce048d1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:43.224030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c5114-3d17-41c4-92f1-dacce048d1e6 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:43.224336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c5114-3d17-41c4-92f1-dacce048d1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:43.224621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c5114-3d17-41c4-92f1-dacce048d1e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:43.236625 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c5114-3d17-41c4-92f1-dacce048d1e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-e23c5114-3d17-41c4-92f1-dacce048d1e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:43.241871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:43.242710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:43.243479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:43.243479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:43.243479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:43.243479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:43.243479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:43.243479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:43.243479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:43.243479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:43.243479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:43.243479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:43.243479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:36:43.243479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:36:43.243479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 34/69] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:43.244105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c5114-3d17-41c4-92f1-dacce048d1e6 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:43.276107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b1276-5779-4c66-b0e6-9f09920ba985 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:43.314427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae198c50-6519-4994-a200-bb9236febd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:43.315056 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae198c50-6519-4994-a200-bb9236febd5b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:43.315346 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae198c50-6519-4994-a200-bb9236febd5b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:43.315621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae198c50-6519-4994-a200-bb9236febd5b None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:43.316381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae198c50-6519-4994-a200-bb9236febd5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:43.316690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae198c50-6519-4994-a200-bb9236febd5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:43.318076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae198c50-6519-4994-a200-bb9236febd5b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:43.328335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 36/70] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:43.328335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae198c50-6519-4994-a200-bb9236febd5b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:43.363342 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:43.364029 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c5114-3d17-41c4-92f1-dacce048d1e6 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:43.389629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:43.400775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8faba-cfd9-4143-9202-32624f452162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:43.401431 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8faba-cfd9-4143-9202-32624f452162 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:43.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8faba-cfd9-4143-9202-32624f452162 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:43.402617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8faba-cfd9-4143-9202-32624f452162 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:43.412910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8faba-cfd9-4143-9202-32624f452162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-caf8faba-cfd9-4143-9202-32624f452162 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:43.418634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:43.419507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:43.420269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:43.420269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:43.420269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:43.420269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:43.420269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:43.420269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:43.420269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:43.420269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:43.420269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:43.420269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:43.420269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:36:43.420269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:36:43.420269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 35/71] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:36:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:43.421510 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85bbf3-7327-4604-adae-2ad9d50f5b4d None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:43.421510 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85bbf3-7327-4604-adae-2ad9d50f5b4d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:43.421651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85bbf3-7327-4604-adae-2ad9d50f5b4d None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:43.423059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 36/72] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:43.423583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85bbf3-7327-4604-adae-2ad9d50f5b4d None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:43.444443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae198c50-6519-4994-a200-bb9236febd5b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:43.477862 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89a2cc-6e38-4fca-8c0f-c4e903231b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:43.478491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89a2cc-6e38-4fca-8c0f-c4e903231b35 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:43.478776 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89a2cc-6e38-4fca-8c0f-c4e903231b35 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:43.479055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89a2cc-6e38-4fca-8c0f-c4e903231b35 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:43.479816 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89a2cc-6e38-4fca-8c0f-c4e903231b35 None admin] RBAC: Adding query filter params (name=member) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:43.480781 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89a2cc-6e38-4fca-8c0f-c4e903231b35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:43.482168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89a2cc-6e38-4fca-8c0f-c4e903231b35 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:43.490456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 37/73] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:36:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:43.491708 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fe682-a10f-479d-90de-62e10e7f4e3c None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:43.492002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fe682-a10f-479d-90de-62e10e7f4e3c None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:43.492293 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fe682-a10f-479d-90de-62e10e7f4e3c None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:43.493531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 38/74] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:43.494232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fe682-a10f-479d-90de-62e10e7f4e3c None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:43.535716 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85bbf3-7327-4604-adae-2ad9d50f5b4d None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:43.571089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915e94c-086e-43fe-97fd-e985a913b0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:43.571737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915e94c-086e-43fe-97fd-e985a913b0fd None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:43.572026 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915e94c-086e-43fe-97fd-e985a913b0fd None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:43.572331 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915e94c-086e-43fe-97fd-e985a913b0fd None admin] PATH_INFO: `/v3/users/b499f31220244d41877273ff02e31ae4` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:43.573541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915e94c-086e-43fe-97fd-e985a913b0fd None admin] RBAC: Authorizing `identity:get_user(user_id=b499f31220244d41877273ff02e31ae4)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:43.575048 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915e94c-086e-43fe-97fd-e985a913b0fd None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:43.576694 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 37/75] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:43 2022] GET /identity/v3/users/b499f31220244d41877273ff02e31ae4 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:43.577819 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72e694-6043-444a-8b9a-d36011363da6 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:43.578109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72e694-6043-444a-8b9a-d36011363da6 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:43.578389 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72e694-6043-444a-8b9a-d36011363da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:43.585677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:43.596881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-cf72e694-6043-444a-8b9a-d36011363da6 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:43.598092 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-cf72e694-6043-444a-8b9a-d36011363da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fe682-a10f-479d-90de-62e10e7f4e3c None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:43.635306 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96c20c-4167-429f-86c8-2f28e97cdc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:43.635970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96c20c-4167-429f-86c8-2f28e97cdc15 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:43.636240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96c20c-4167-429f-86c8-2f28e97cdc15 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:43.636554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96c20c-4167-429f-86c8-2f28e97cdc15 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:43.637289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96c20c-4167-429f-86c8-2f28e97cdc15 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:43.637595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96c20c-4167-429f-86c8-2f28e97cdc15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:43.639231 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96c20c-4167-429f-86c8-2f28e97cdc15 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:43.646709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:43.647627 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 39/76] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:36:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:43.648546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96c20c-4167-429f-86c8-2f28e97cdc15 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:43.672412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 38/77] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:43.673731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f3a4d-3f46-47b7-82f0-bd45f7f201b7 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:43.674030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f3a4d-3f46-47b7-82f0-bd45f7f201b7 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:43.674334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f3a4d-3f46-47b7-82f0-bd45f7f201b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:43.692119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-9b5f3a4d-3f46-47b7-82f0-bd45f7f201b7 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:43.693335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-9b5f3a4d-3f46-47b7-82f0-bd45f7f201b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:43.754143 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96c20c-4167-429f-86c8-2f28e97cdc15 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:43.761146 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 39/78] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:43.762530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f3a4d-3f46-47b7-82f0-bd45f7f201b7 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:43.793492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef157da-e460-43f3-a579-730b3e165c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:43.794153 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef157da-e460-43f3-a579-730b3e165c86 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:43.794435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef157da-e460-43f3-a579-730b3e165c86 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:43.794713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef157da-e460-43f3-a579-730b3e165c86 None admin] PATH_INFO: `/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:43.810737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef157da-e460-43f3-a579-730b3e165c86 None admin] RBAC: Authorizing `identity:get_user(user_id=67faf276eb0a4b9baf8b596eb39d6bd2)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:43.812199 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef157da-e460-43f3-a579-730b3e165c86 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:43.813880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 40/79] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:43 2022] GET /identity/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:43.814651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef157da-e460-43f3-a579-730b3e165c86 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:43.828611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:43.866511 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f3a4d-3f46-47b7-82f0-bd45f7f201b7 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:43.881460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:43.903232 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec9304-c906-485f-b07c-ad1e52d050c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:43.903859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec9304-c906-485f-b07c-ad1e52d050c7 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:43.904139 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec9304-c906-485f-b07c-ad1e52d050c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:43.904452 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec9304-c906-485f-b07c-ad1e52d050c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:43.919938 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef157da-e460-43f3-a579-730b3e165c86 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:43.954653 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b9ba3-6652-4a87-846e-5593fe91df82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:43.955307 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b9ba3-6652-4a87-846e-5593fe91df82 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:43.955590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b9ba3-6652-4a87-846e-5593fe91df82 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:43.955866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b9ba3-6652-4a87-846e-5593fe91df82 None admin] PATH_INFO: `/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:43.957079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b9ba3-6652-4a87-846e-5593fe91df82 None admin] RBAC: Authorizing `identity:get_user(user_id=67faf276eb0a4b9baf8b596eb39d6bd2)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:43.958517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b9ba3-6652-4a87-846e-5593fe91df82 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:43.960131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 41/80] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:43 2022] GET /identity/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:43.960928 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b9ba3-6652-4a87-846e-5593fe91df82 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:43.977704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec9304-c906-485f-b07c-ad1e52d050c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:43.979192 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec9304-c906-485f-b07c-ad1e52d050c7 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:44.039602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:44.063956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b9ba3-6652-4a87-846e-5593fe91df82 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:44.081670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 40/81] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:36:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:36:44.082521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec9304-c906-485f-b07c-ad1e52d050c7 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:44.096142 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:44.101011 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc506a0f-3e49-49d7-a50e-cf646e675b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:44.101665 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc506a0f-3e49-49d7-a50e-cf646e675b6e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:44.101950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc506a0f-3e49-49d7-a50e-cf646e675b6e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:44.102228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc506a0f-3e49-49d7-a50e-cf646e675b6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:44.112900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc506a0f-3e49-49d7-a50e-cf646e675b6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-fc506a0f-3e49-49d7-a50e-cf646e675b6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:44.118317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:44.119157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:44.119919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:44.119919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:44.119919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:44.119919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:44.119919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:44.119919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:44.119919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:44.119919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:44.119919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:44.119919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:44.119919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:36:44.119919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:36:44.119919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 42/82] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:44.125678 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc506a0f-3e49-49d7-a50e-cf646e675b6e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:44.185077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec9304-c906-485f-b07c-ad1e52d050c7 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:44.218955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17f8c2-b4a8-48df-8dbe-fb56a43293c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:44.219602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17f8c2-b4a8-48df-8dbe-fb56a43293c2 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:44.219888 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17f8c2-b4a8-48df-8dbe-fb56a43293c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:44.220156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17f8c2-b4a8-48df-8dbe-fb56a43293c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:44.221349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17f8c2-b4a8-48df-8dbe-fb56a43293c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:44.222827 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17f8c2-b4a8-48df-8dbe-fb56a43293c2 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:44.224420 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 41/83] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:36:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:44.225187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17f8c2-b4a8-48df-8dbe-fb56a43293c2 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:44.225968 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc506a0f-3e49-49d7-a50e-cf646e675b6e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:44.263826 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203ba5b-4910-488d-801a-13508119b15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:44.264491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203ba5b-4910-488d-801a-13508119b15e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:44.264835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203ba5b-4910-488d-801a-13508119b15e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:44.265116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203ba5b-4910-488d-801a-13508119b15e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:44.268607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:44.308669 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:44.328519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17f8c2-b4a8-48df-8dbe-fb56a43293c2 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:44.334084 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203ba5b-4910-488d-801a-13508119b15e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:44.335575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203ba5b-4910-488d-801a-13508119b15e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:44.362901 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e0d6e-dfac-405e-96be-5c26c548fe77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:44.363529 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e0d6e-dfac-405e-96be-5c26c548fe77 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:44.363814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e0d6e-dfac-405e-96be-5c26c548fe77 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:44.364095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e0d6e-dfac-405e-96be-5c26c548fe77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:44.432865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e0d6e-dfac-405e-96be-5c26c548fe77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:44.434441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e0d6e-dfac-405e-96be-5c26c548fe77 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:44.436657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 43/84] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:36:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:36:44.437507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203ba5b-4910-488d-801a-13508119b15e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:44.473602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:44.521247 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:44.542301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203ba5b-4910-488d-801a-13508119b15e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:44.542495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 42/85] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:36:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:36:44.543397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e0d6e-dfac-405e-96be-5c26c548fe77 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:44.582375 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3ebde-4ed2-4b48-9110-735e323b09d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:44.582954 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3ebde-4ed2-4b48-9110-735e323b09d7 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:44.583235 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3ebde-4ed2-4b48-9110-735e323b09d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:44.583508 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3ebde-4ed2-4b48-9110-735e323b09d7 None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:44.584712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3ebde-4ed2-4b48-9110-735e323b09d7 None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:44.586156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3ebde-4ed2-4b48-9110-735e323b09d7 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:44.587810 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 44/86] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:44 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:44.588578 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3ebde-4ed2-4b48-9110-735e323b09d7 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:44.647364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e0d6e-dfac-405e-96be-5c26c548fe77 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:44.681225 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcb1b1-c340-40a7-ac96-c0a3668046c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:44.681855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcb1b1-c340-40a7-ac96-c0a3668046c7 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:44.682137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcb1b1-c340-40a7-ac96-c0a3668046c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:44.682417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcb1b1-c340-40a7-ac96-c0a3668046c7 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:44.683186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dcb1b1-c340-40a7-ac96-c0a3668046c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:44.683579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dcb1b1-c340-40a7-ac96-c0a3668046c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:44.685023 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dcb1b1-c340-40a7-ac96-c0a3668046c7 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:44.692210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3ebde-4ed2-4b48-9110-735e323b09d7 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:44.693229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 43/87] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:36:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:44.693864 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcb1b1-c340-40a7-ac96-c0a3668046c7 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:44.699234 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:44.732506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983859b1-0eb3-4f3c-a66c-93899047cece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:44.733151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983859b1-0eb3-4f3c-a66c-93899047cece None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:44.733436 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983859b1-0eb3-4f3c-a66c-93899047cece None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:44.733712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983859b1-0eb3-4f3c-a66c-93899047cece None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:44.734404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983859b1-0eb3-4f3c-a66c-93899047cece None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:44.735883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983859b1-0eb3-4f3c-a66c-93899047cece None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:44.738385 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:44.761763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 45/88] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:36:44 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:36:44.763470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983859b1-0eb3-4f3c-a66c-93899047cece None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:44.833232 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcb1b1-c340-40a7-ac96-c0a3668046c7 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:44.872769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857cf8d-6b5d-4f65-9caa-5ef118eddf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:44.873822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857cf8d-6b5d-4f65-9caa-5ef118eddf1c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:44.874304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857cf8d-6b5d-4f65-9caa-5ef118eddf1c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:44.874765 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857cf8d-6b5d-4f65-9caa-5ef118eddf1c None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:44.876813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857cf8d-6b5d-4f65-9caa-5ef118eddf1c None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:44.879184 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857cf8d-6b5d-4f65-9caa-5ef118eddf1c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:44.887561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 44/89] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:44 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:44.887561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857cf8d-6b5d-4f65-9caa-5ef118eddf1c None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:44.909874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983859b1-0eb3-4f3c-a66c-93899047cece None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:44.964367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edaa07-71cf-4f20-b2a7-b2205d9c1dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:44.965223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edaa07-71cf-4f20-b2a7-b2205d9c1dae None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:44.967654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edaa07-71cf-4f20-b2a7-b2205d9c1dae None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:44.967654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edaa07-71cf-4f20-b2a7-b2205d9c1dae None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:44.967654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74edaa07-71cf-4f20-b2a7-b2205d9c1dae None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:44.968527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74edaa07-71cf-4f20-b2a7-b2205d9c1dae None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:44.970243 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 46/90] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:44 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:44.971018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edaa07-71cf-4f20-b2a7-b2205d9c1dae None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:45.004314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:45.031522 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857cf8d-6b5d-4f65-9caa-5ef118eddf1c None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:45.044776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:45.080007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cf59b-de71-4fde-8884-6e6bf44bc7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:45.080877 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cf59b-de71-4fde-8884-6e6bf44bc7eb None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:45.081175 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cf59b-de71-4fde-8884-6e6bf44bc7eb None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:45.081484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cf59b-de71-4fde-8884-6e6bf44bc7eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:45.082302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497cf59b-de71-4fde-8884-6e6bf44bc7eb None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=42291678aa1842ff9db31b9d1e39dc2c, user.id=b499f31220244d41877273ff02e31ae4) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:45.082627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497cf59b-de71-4fde-8884-6e6bf44bc7eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:45.084139 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497cf59b-de71-4fde-8884-6e6bf44bc7eb None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:45.094037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 45/91] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:36:44 2022] GET /identity/v3/role_assignments?user.id=b499f31220244d41877273ff02e31ae4&scope.project.id=42291678aa1842ff9db31b9d1e39dc2c&role.id=19557e941d6f475ba92cf862846f9872 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:45.094037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cf59b-de71-4fde-8884-6e6bf44bc7eb None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:45.094288 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edaa07-71cf-4f20-b2a7-b2205d9c1dae None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:45.135606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89840f8d-a34b-4ca4-924b-6985e857a07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:45.136525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89840f8d-a34b-4ca4-924b-6985e857a07e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:45.136821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89840f8d-a34b-4ca4-924b-6985e857a07e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:45.137188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89840f8d-a34b-4ca4-924b-6985e857a07e None admin] PATH_INFO: `/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:45.138469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89840f8d-a34b-4ca4-924b-6985e857a07e None admin] RBAC: Authorizing `identity:get_user(user_id=67faf276eb0a4b9baf8b596eb39d6bd2)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:45.140080 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89840f8d-a34b-4ca4-924b-6985e857a07e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:45.141838 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 47/92] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:44 2022] GET /identity/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:45.142635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89840f8d-a34b-4ca4-924b-6985e857a07e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:45.224946 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cf59b-de71-4fde-8884-6e6bf44bc7eb None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:45.256243 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:45.264056 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89840f8d-a34b-4ca4-924b-6985e857a07e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:45.266940 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749ef82-48ce-49bd-a1bd-39f8c926719c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:45.267598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749ef82-48ce-49bd-a1bd-39f8c926719c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:45.267888 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749ef82-48ce-49bd-a1bd-39f8c926719c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:45.268169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749ef82-48ce-49bd-a1bd-39f8c926719c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:45.269033 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749ef82-48ce-49bd-a1bd-39f8c926719c None admin] RBAC: Adding query filter params (role.id=f3b7ffe4d1f04af18f7da8d9a51031b3, scope.project.id=42291678aa1842ff9db31b9d1e39dc2c, user.id=67faf276eb0a4b9baf8b596eb39d6bd2) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:45.269350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749ef82-48ce-49bd-a1bd-39f8c926719c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:45.270854 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749ef82-48ce-49bd-a1bd-39f8c926719c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:45.278473 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 46/93] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:36:45 2022] GET /identity/v3/role_assignments?user.id=67faf276eb0a4b9baf8b596eb39d6bd2&scope.project.id=42291678aa1842ff9db31b9d1e39dc2c&role.id=f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:45.279212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749ef82-48ce-49bd-a1bd-39f8c926719c None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:45.325968 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816aef6-9109-4373-8b2a-c13cb0c70a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:45.326632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816aef6-9109-4373-8b2a-c13cb0c70a09 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:45.326920 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816aef6-9109-4373-8b2a-c13cb0c70a09 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:45.327201 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816aef6-9109-4373-8b2a-c13cb0c70a09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:45.328057 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816aef6-9109-4373-8b2a-c13cb0c70a09 None admin] RBAC: Adding query filter params (role.id=548e622a9ae745d99d4cd34e68c1652c, scope.project.id=42291678aa1842ff9db31b9d1e39dc2c, user.id=67faf276eb0a4b9baf8b596eb39d6bd2) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:45.332812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816aef6-9109-4373-8b2a-c13cb0c70a09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:45.333562 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816aef6-9109-4373-8b2a-c13cb0c70a09 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:45.337328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:45.348531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 48/94] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:36:45 2022] GET /identity/v3/role_assignments?user.id=67faf276eb0a4b9baf8b596eb39d6bd2&scope.project.id=42291678aa1842ff9db31b9d1e39dc2c&role.id=548e622a9ae745d99d4cd34e68c1652c => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:45.421737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749ef82-48ce-49bd-a1bd-39f8c926719c None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:45.463850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4f2f8-0c9f-473e-9709-837ff47015e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:45.464608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4f2f8-0c9f-473e-9709-837ff47015e0 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:45.465298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4f2f8-0c9f-473e-9709-837ff47015e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:45.465298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4f2f8-0c9f-473e-9709-837ff47015e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:45.546041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4f2f8-0c9f-473e-9709-837ff47015e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:45.547611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4f2f8-0c9f-473e-9709-837ff47015e0 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:45.591552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:45.659656 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 47/95] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:36:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:36:45.665896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816aef6-9109-4373-8b2a-c13cb0c70a09 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:45.797454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816aef6-9109-4373-8b2a-c13cb0c70a09 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:45.836402 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5519542-ec51-4e34-b50f-4b14dab6a000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:45.837127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5519542-ec51-4e34-b50f-4b14dab6a000 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:45.837422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5519542-ec51-4e34-b50f-4b14dab6a000 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:45.837704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5519542-ec51-4e34-b50f-4b14dab6a000 None admin] PATH_INFO: `/v3/projects/f91e09770ccc4ff8813c6cfe33d2bffe` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:45.838896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5519542-ec51-4e34-b50f-4b14dab6a000 None admin] RBAC: Authorizing `identity:get_project(project_id=f91e09770ccc4ff8813c6cfe33d2bffe)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:45.840852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5519542-ec51-4e34-b50f-4b14dab6a000 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:45.843095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 49/96] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:45 2022] GET /identity/v3/projects/f91e09770ccc4ff8813c6cfe33d2bffe => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:45.849218 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4f2f8-0c9f-473e-9709-837ff47015e0 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:45.959289 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4f2f8-0c9f-473e-9709-837ff47015e0 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:45.996322 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0436c16-d43a-4622-83ed-f874921f2788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:45.997400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0436c16-d43a-4622-83ed-f874921f2788 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:45.997694 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0436c16-d43a-4622-83ed-f874921f2788 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:45.997975 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0436c16-d43a-4622-83ed-f874921f2788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:45.998850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0436c16-d43a-4622-83ed-f874921f2788 None admin] RBAC: Adding query filter params (role.id=f3b7ffe4d1f04af18f7da8d9a51031b3, scope.project.id=f91e09770ccc4ff8813c6cfe33d2bffe, user.id=67faf276eb0a4b9baf8b596eb39d6bd2) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:45.999178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0436c16-d43a-4622-83ed-f874921f2788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:46.000683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0436c16-d43a-4622-83ed-f874921f2788 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:46.007087 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 48/97] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:36:45 2022] GET /identity/v3/role_assignments?user.id=67faf276eb0a4b9baf8b596eb39d6bd2&scope.project.id=f91e09770ccc4ff8813c6cfe33d2bffe&role.id=f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:47.764924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc8e2f-807c-40b7-a02c-5458b6526ab4 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:47.765489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc8e2f-807c-40b7-a02c-5458b6526ab4 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:47.765489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc8e2f-807c-40b7-a02c-5458b6526ab4 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:47.766882 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 50/98] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:47.772715 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aeda05-f3d9-4722-ad47-52e8714d4d34 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:47.773230 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aeda05-f3d9-4722-ad47-52e8714d4d34 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:47.773455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aeda05-f3d9-4722-ad47-52e8714d4d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:47.787658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:47.805664 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-37aeda05-f3d9-4722-ad47-52e8714d4d34 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:47.806866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-37aeda05-f3d9-4722-ad47-52e8714d4d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:47.961817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 49/99] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:47.982431 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891a257-6e43-4172-8a0b-757a0a3976a2 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:47.983009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891a257-6e43-4172-8a0b-757a0a3976a2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:47.983467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891a257-6e43-4172-8a0b-757a0a3976a2 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:47.985071 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 51/100] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:47.990727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d56e5-bac0-474a-8525-07a56a9eabcd None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:47.991255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d56e5-bac0-474a-8525-07a56a9eabcd None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:47.992354 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891a257-6e43-4172-8a0b-757a0a3976a2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:47.993611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d56e5-bac0-474a-8525-07a56a9eabcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:48.015099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-915d56e5-bac0-474a-8525-07a56a9eabcd None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:48.016605 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-915d56e5-bac0-474a-8525-07a56a9eabcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:48.068926 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:48.133811 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 50/101] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:48.135119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541edf3b-df40-4332-a9d9-95b25349ea43 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:48.135418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541edf3b-df40-4332-a9d9-95b25349ea43 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:48.135703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541edf3b-df40-4332-a9d9-95b25349ea43 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:48.137397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 51/102] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:48.143038 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6248d1e-1ab3-401d-acc3-3bca7542610e None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:48.143336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6248d1e-1ab3-401d-acc3-3bca7542610e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:48.143623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6248d1e-1ab3-401d-acc3-3bca7542610e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:48.163564 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-b6248d1e-1ab3-401d-acc3-3bca7542610e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:48.164818 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b6248d1e-1ab3-401d-acc3-3bca7542610e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:48.235685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:48.258441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 52/103] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:36:48.260601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6248d1e-1ab3-401d-acc3-3bca7542610e None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:48.285113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891a257-6e43-4172-8a0b-757a0a3976a2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:48.331038 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf53eb-8403-4e2c-a0fb-777c765d7313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:48.331986 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf53eb-8403-4e2c-a0fb-777c765d7313 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:48.332458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf53eb-8403-4e2c-a0fb-777c765d7313 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:48.332890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf53eb-8403-4e2c-a0fb-777c765d7313 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:48.334233 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf53eb-8403-4e2c-a0fb-777c765d7313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:48.335830 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf53eb-8403-4e2c-a0fb-777c765d7313 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:48.337709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 52/104] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:36:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:48.338649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf53eb-8403-4e2c-a0fb-777c765d7313 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:48.397691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6248d1e-1ab3-401d-acc3-3bca7542610e None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:48.439665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca334223-22be-42b9-8e64-2307273b04b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:48.440223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca334223-22be-42b9-8e64-2307273b04b9 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:48.440562 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca334223-22be-42b9-8e64-2307273b04b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:48.440846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca334223-22be-42b9-8e64-2307273b04b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:48.445256 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:48.452546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf53eb-8403-4e2c-a0fb-777c765d7313 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:48.457318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca334223-22be-42b9-8e64-2307273b04b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-ca334223-22be-42b9-8e64-2307273b04b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:48.463263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:48.464188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:48.465405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:48.465405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:48.465405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:48.465405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:48.465405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:48.465405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:48.465405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:48.465405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:48.465405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:48.465405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:48.465405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:36:48.465405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:36:48.465405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 53/105] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:36:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:48.466035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca334223-22be-42b9-8e64-2307273b04b9 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:48.498060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb3926-8863-4ce8-bd2c-4deadf1574e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:48.499206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb3926-8863-4ce8-bd2c-4deadf1574e6 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:48.499814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb3926-8863-4ce8-bd2c-4deadf1574e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:48.500335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb3926-8863-4ce8-bd2c-4deadf1574e6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:48.513146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb3926-8863-4ce8-bd2c-4deadf1574e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-25fb3926-8863-4ce8-bd2c-4deadf1574e6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:48.518983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:48.519873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:48.520994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:48.520994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:48.520994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:48.520994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:48.520994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:48.520994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:48.520994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:48.520994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:48.520994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:48.520994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:48.520994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:36:48.520994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:36:48.523165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 53/106] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:48.524251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb3926-8863-4ce8-bd2c-4deadf1574e6 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:48.531685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:48.628210 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca334223-22be-42b9-8e64-2307273b04b9 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:48.675785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713929ea-cd0c-4cbc-9795-90f1d22b28a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:48.676846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713929ea-cd0c-4cbc-9795-90f1d22b28a6 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:48.677379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713929ea-cd0c-4cbc-9795-90f1d22b28a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:48.677848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713929ea-cd0c-4cbc-9795-90f1d22b28a6 None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:48.678741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713929ea-cd0c-4cbc-9795-90f1d22b28a6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:48.680483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713929ea-cd0c-4cbc-9795-90f1d22b28a6 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:48.704701 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb3926-8863-4ce8-bd2c-4deadf1574e6 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:48.717138 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 54/107] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Aug 3 18:36:48 2022] POST /identity/v3/projects => generated 302 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:36:48.718296 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713929ea-cd0c-4cbc-9795-90f1d22b28a6 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:48.805125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd96ec-5be6-49ea-b84b-d28dc09cf882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:48.805821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd96ec-5be6-49ea-b84b-d28dc09cf882 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:48.806065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd96ec-5be6-49ea-b84b-d28dc09cf882 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:48.806348 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd96ec-5be6-49ea-b84b-d28dc09cf882 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:48.807107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd96ec-5be6-49ea-b84b-d28dc09cf882 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:48.807449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd96ec-5be6-49ea-b84b-d28dc09cf882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:48.808976 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd96ec-5be6-49ea-b84b-d28dc09cf882 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:48.830768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 54/108] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:48.831864 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dc486-b6d7-4e14-963c-688dfe77f89c None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:48.832157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dc486-b6d7-4e14-963c-688dfe77f89c None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:48.832461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dc486-b6d7-4e14-963c-688dfe77f89c None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:48.833706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 55/109] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:48.838992 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795b3e9-8ca2-4edd-a721-e2f7bdcf7ba8 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:48.839951 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795b3e9-8ca2-4edd-a721-e2f7bdcf7ba8 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:48.840272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795b3e9-8ca2-4edd-a721-e2f7bdcf7ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:48.860730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713929ea-cd0c-4cbc-9795-90f1d22b28a6 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:48.884769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:48.897546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-c795b3e9-8ca2-4edd-a721-e2f7bdcf7ba8 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:48.898720 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c795b3e9-8ca2-4edd-a721-e2f7bdcf7ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:48.903789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9032a7-1677-4cda-9459-579181dd5a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:48.903789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9032a7-1677-4cda-9459-579181dd5a67 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:48.903789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9032a7-1677-4cda-9459-579181dd5a67 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:48.903789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9032a7-1677-4cda-9459-579181dd5a67 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:48.906853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9032a7-1677-4cda-9459-579181dd5a67 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:48.906853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9032a7-1677-4cda-9459-579181dd5a67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:48.907516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9032a7-1677-4cda-9459-579181dd5a67 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:48.923068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 55/110] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:36:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:48.924229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9032a7-1677-4cda-9459-579181dd5a67 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:48.927981 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:49.003683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 56/111] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:49.008319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795b3e9-8ca2-4edd-a721-e2f7bdcf7ba8 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:49.061737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9032a7-1677-4cda-9459-579181dd5a67 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:49.099946 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f11f8b-38ff-4f30-9017-96467197a771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:49.100952 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f11f8b-38ff-4f30-9017-96467197a771 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:49.101399 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f11f8b-38ff-4f30-9017-96467197a771 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:49.101677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f11f8b-38ff-4f30-9017-96467197a771 None admin] PATH_INFO: `/v3/users/b499f31220244d41877273ff02e31ae4` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:49.102883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f11f8b-38ff-4f30-9017-96467197a771 None admin] RBAC: Authorizing `identity:get_user(user_id=b499f31220244d41877273ff02e31ae4)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:49.104364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f11f8b-38ff-4f30-9017-96467197a771 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:49.111255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 56/112] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:48 2022] GET /identity/v3/users/b499f31220244d41877273ff02e31ae4 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:49.111255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544cfe0-fcd9-4520-b26b-878cb3af96fa None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:49.111255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544cfe0-fcd9-4520-b26b-878cb3af96fa None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:49.111255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544cfe0-fcd9-4520-b26b-878cb3af96fa None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:49.111255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 57/113] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:49.111255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544cfe0-fcd9-4520-b26b-878cb3af96fa None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:49.133626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795b3e9-8ca2-4edd-a721-e2f7bdcf7ba8 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:49.149077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:49.173386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8bf96-2815-4285-a9c5-6ff0028361fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:49.174288 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8bf96-2815-4285-a9c5-6ff0028361fc None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:49.174742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8bf96-2815-4285-a9c5-6ff0028361fc None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:49.175171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8bf96-2815-4285-a9c5-6ff0028361fc None admin] PATH_INFO: `/v3/users/af8505067983430097b469a5330dbcb3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:49.191154 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8bf96-2815-4285-a9c5-6ff0028361fc None admin] RBAC: Authorizing `identity:get_user(user_id=af8505067983430097b469a5330dbcb3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:49.192671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8bf96-2815-4285-a9c5-6ff0028361fc None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:49.194571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 57/114] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:49 2022] GET /identity/v3/users/af8505067983430097b469a5330dbcb3 => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:49.200769 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac972f-a6c7-4732-8c9e-b74139d48c6b None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:49.201065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac972f-a6c7-4732-8c9e-b74139d48c6b None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:49.201351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac972f-a6c7-4732-8c9e-b74139d48c6b None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:49.202688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 58/115] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:49.203410 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac972f-a6c7-4732-8c9e-b74139d48c6b None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:49.213363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:49.244265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544cfe0-fcd9-4520-b26b-878cb3af96fa None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:49.288253 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981745d-4aff-440c-b91a-058bbf2b0d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:49.289208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981745d-4aff-440c-b91a-058bbf2b0d3f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:49.289498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981745d-4aff-440c-b91a-058bbf2b0d3f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:49.289778 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981745d-4aff-440c-b91a-058bbf2b0d3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:49.301113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981745d-4aff-440c-b91a-058bbf2b0d3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-1981745d-4aff-440c-b91a-058bbf2b0d3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:49.306621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:49.307498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:49.313632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:49.313632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:49.313632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:49.313632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:49.313632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:49.313632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:49.313632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:49.313632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:49.313632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:49.313632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:49.313632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:36:49.313632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:36:49.313632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 58/116] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:49.313632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790b3a5-97e3-4ecf-87bf-d823a48fbce8 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:49.314350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790b3a5-97e3-4ecf-87bf-d823a48fbce8 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:49.314350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790b3a5-97e3-4ecf-87bf-d823a48fbce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:49.317734 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac972f-a6c7-4732-8c9e-b74139d48c6b None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:49.332741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-3790b3a5-97e3-4ecf-87bf-d823a48fbce8 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:49.333941 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-3790b3a5-97e3-4ecf-87bf-d823a48fbce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:49.374848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19496696-9d08-4da3-bb0c-18d20549dc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:49.375512 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19496696-9d08-4da3-bb0c-18d20549dc21 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:49.375795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19496696-9d08-4da3-bb0c-18d20549dc21 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:49.376072 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19496696-9d08-4da3-bb0c-18d20549dc21 None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:49.377658 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19496696-9d08-4da3-bb0c-18d20549dc21 None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:49.378752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19496696-9d08-4da3-bb0c-18d20549dc21 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:49.380892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 59/117] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:49 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:49.381598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19496696-9d08-4da3-bb0c-18d20549dc21 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:49.430963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 59/118] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:49.432074 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6794e-a5f3-4f26-9645-44bd94c956a4 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:49.432400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6794e-a5f3-4f26-9645-44bd94c956a4 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:49.432681 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6794e-a5f3-4f26-9645-44bd94c956a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:49.458078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-6bd6794e-a5f3-4f26-9645-44bd94c956a4 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:49.459274 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-6bd6794e-a5f3-4f26-9645-44bd94c956a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:49.474677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:49.479346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:49.507133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19496696-9d08-4da3-bb0c-18d20549dc21 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:49.536430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 60/119] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:49.548470 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6794e-a5f3-4f26-9645-44bd94c956a4 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:49.563238 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc37c33-c550-41b0-94ef-364b8bd82c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:49.564112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc37c33-c550-41b0-94ef-364b8bd82c48 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:49.564508 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc37c33-c550-41b0-94ef-364b8bd82c48 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:49.564875 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc37c33-c550-41b0-94ef-364b8bd82c48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:49.641960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc37c33-c550-41b0-94ef-364b8bd82c48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:49.643441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc37c33-c550-41b0-94ef-364b8bd82c48 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:49.654289 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6794e-a5f3-4f26-9645-44bd94c956a4 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:49.691174 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7172e6-9be5-484f-ac7b-b246209fd973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:49.692011 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7172e6-9be5-484f-ac7b-b246209fd973 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:49.692326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7172e6-9be5-484f-ac7b-b246209fd973 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:49.692728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7172e6-9be5-484f-ac7b-b246209fd973 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:49.693472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7172e6-9be5-484f-ac7b-b246209fd973 None admin] RBAC: Adding query filter params (name=member) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:49.693783 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7172e6-9be5-484f-ac7b-b246209fd973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:49.695260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7172e6-9be5-484f-ac7b-b246209fd973 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:49.706030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 61/120] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:36:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:49.706621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7172e6-9be5-484f-ac7b-b246209fd973 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:49.726632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:49.731991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:49.761935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 60/121] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:36:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:36:49.762908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc37c33-c550-41b0-94ef-364b8bd82c48 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:49.823764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7172e6-9be5-484f-ac7b-b246209fd973 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:49.858830 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01f6d3-7abc-4047-b63f-8d648cb1304e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:49.859777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01f6d3-7abc-4047-b63f-8d648cb1304e None admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:49.860130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01f6d3-7abc-4047-b63f-8d648cb1304e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:49.860509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01f6d3-7abc-4047-b63f-8d648cb1304e None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c/users/b499f31220244d41877273ff02e31ae4/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:49.862703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01f6d3-7abc-4047-b63f-8d648cb1304e None admin] RBAC: Authorizing `identity:create_grant(project_id=42291678aa1842ff9db31b9d1e39dc2c, user_id=b499f31220244d41877273ff02e31ae4, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:49.864178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01f6d3-7abc-4047-b63f-8d648cb1304e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:49.873227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc37c33-c550-41b0-94ef-364b8bd82c48 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:49.881792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 62/122] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:36:49 2022] PUT /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c/users/b499f31220244d41877273ff02e31ae4/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:36:49.882788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01f6d3-7abc-4047-b63f-8d648cb1304e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:49.929352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a5f16-c748-4ca7-9aef-b7d6cb0511c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:49.930503 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a5f16-c748-4ca7-9aef-b7d6cb0511c8 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:49.931437 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a5f16-c748-4ca7-9aef-b7d6cb0511c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:49.931859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a5f16-c748-4ca7-9aef-b7d6cb0511c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:49.954899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a5f16-c748-4ca7-9aef-b7d6cb0511c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-5a6a5f16-c748-4ca7-9aef-b7d6cb0511c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:49.961429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:49.962523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:49.963515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:49.963515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:49.963515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:49.963515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:49.963515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:49.963515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:49.963515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:49.963515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:49.963515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:49.963515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:49.963515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:36:49.963515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:36:49.966164 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 61/123] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:36:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:49.967344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a5f16-c748-4ca7-9aef-b7d6cb0511c8 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:50.006968 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:50.024746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:50.043836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01f6d3-7abc-4047-b63f-8d648cb1304e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:50.088649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469102d7-35b0-45ef-9d8f-75421b48b91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:50.089416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469102d7-35b0-45ef-9d8f-75421b48b91a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:50.089416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469102d7-35b0-45ef-9d8f-75421b48b91a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:50.090018 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469102d7-35b0-45ef-9d8f-75421b48b91a None admin] PATH_INFO: `/v3/roles/member` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:50.102525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469102d7-35b0-45ef-9d8f-75421b48b91a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-469102d7-35b0-45ef-9d8f-75421b48b91a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:50.108764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:50.109640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:50.110410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:50.110410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:50.110410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:50.110410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:50.110410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:50.110410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:50.110410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:50.110410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:50.110410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:50.110410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:50.110410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:36:50.110410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:36:50.110410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 63/124] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:50.111072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469102d7-35b0-45ef-9d8f-75421b48b91a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:50.130536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a5f16-c748-4ca7-9aef-b7d6cb0511c8 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:50.179787 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88447d-ba90-4d05-9243-955505aa25ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:50.180890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88447d-ba90-4d05-9243-955505aa25ca None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:50.181319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88447d-ba90-4d05-9243-955505aa25ca None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:50.181745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88447d-ba90-4d05-9243-955505aa25ca None admin] PATH_INFO: `/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:50.183611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f88447d-ba90-4d05-9243-955505aa25ca None admin] RBAC: Authorizing `identity:get_user(user_id=67faf276eb0a4b9baf8b596eb39d6bd2)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:50.185563 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f88447d-ba90-4d05-9243-955505aa25ca None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:50.187839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 62/125] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:49 2022] GET /identity/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2 => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:50.189318 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88447d-ba90-4d05-9243-955505aa25ca None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:50.225240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469102d7-35b0-45ef-9d8f-75421b48b91a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:50.265021 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21e4b5-d7f7-40d7-a31c-402dd0d589d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:50.265832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21e4b5-d7f7-40d7-a31c-402dd0d589d7 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:50.266190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21e4b5-d7f7-40d7-a31c-402dd0d589d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:50.266536 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21e4b5-d7f7-40d7-a31c-402dd0d589d7 None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:50.267956 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21e4b5-d7f7-40d7-a31c-402dd0d589d7 None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:50.269613 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21e4b5-d7f7-40d7-a31c-402dd0d589d7 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:50.271579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 64/126] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:50 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:50.272613 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21e4b5-d7f7-40d7-a31c-402dd0d589d7 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:50.279495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:50.336517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88447d-ba90-4d05-9243-955505aa25ca None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:50.339734 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:50.383601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa3aec-7ac1-4077-ab40-39d09d0b84c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:50.384271 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa3aec-7ac1-4077-ab40-39d09d0b84c4 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:50.384601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa3aec-7ac1-4077-ab40-39d09d0b84c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:50.384884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa3aec-7ac1-4077-ab40-39d09d0b84c4 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:50.385621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa3aec-7ac1-4077-ab40-39d09d0b84c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:50.386043 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa3aec-7ac1-4077-ab40-39d09d0b84c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:50.388326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa3aec-7ac1-4077-ab40-39d09d0b84c4 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:50.400267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 63/127] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:36:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:50.401173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa3aec-7ac1-4077-ab40-39d09d0b84c4 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:50.402075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21e4b5-d7f7-40d7-a31c-402dd0d589d7 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:50.448455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447c288-ca59-4e37-8a84-fc382ba9a282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:50.450608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447c288-ca59-4e37-8a84-fc382ba9a282 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:50.451107 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447c288-ca59-4e37-8a84-fc382ba9a282 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:50.451673 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447c288-ca59-4e37-8a84-fc382ba9a282 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:50.452644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447c288-ca59-4e37-8a84-fc382ba9a282 None admin] RBAC: Adding query filter params (name=member) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:50.453151 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447c288-ca59-4e37-8a84-fc382ba9a282 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:50.455406 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447c288-ca59-4e37-8a84-fc382ba9a282 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:50.468217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 65/128] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:36:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:50.469522 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447c288-ca59-4e37-8a84-fc382ba9a282 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:50.526390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa3aec-7ac1-4077-ab40-39d09d0b84c4 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:50.553238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:50.566381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bffb0-13b0-4898-bf0d-d4281c60bd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:50.567055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bffb0-13b0-4898-bf0d-d4281c60bd76 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:50.567345 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bffb0-13b0-4898-bf0d-d4281c60bd76 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:50.567638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bffb0-13b0-4898-bf0d-d4281c60bd76 None admin] PATH_INFO: `/v3/projects/f91e09770ccc4ff8813c6cfe33d2bffe` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:50.569398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9bffb0-13b0-4898-bf0d-d4281c60bd76 None admin] RBAC: Authorizing `identity:get_project(project_id=f91e09770ccc4ff8813c6cfe33d2bffe)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:50.571545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9bffb0-13b0-4898-bf0d-d4281c60bd76 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:50.574373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 64/129] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:50 2022] GET /identity/v3/projects/f91e09770ccc4ff8813c6cfe33d2bffe => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:50.579477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bffb0-13b0-4898-bf0d-d4281c60bd76 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:50.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447c288-ca59-4e37-8a84-fc382ba9a282 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:50.617986 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:50.645284 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9a080-7c0f-41d7-9737-4785badf30a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:50.646441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9a080-7c0f-41d7-9737-4785badf30a8 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:50.646866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9a080-7c0f-41d7-9737-4785badf30a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:50.647306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9a080-7c0f-41d7-9737-4785badf30a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:50.648255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9a080-7c0f-41d7-9737-4785badf30a8 None admin] RBAC: Adding query filter params (role.id=5f5e6a876c4a4fb0869e5bea07e7a65c, scope.project.id=42291678aa1842ff9db31b9d1e39dc2c, user.id=af8505067983430097b469a5330dbcb3) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:50.648869 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9a080-7c0f-41d7-9737-4785badf30a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:50.651213 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9a080-7c0f-41d7-9737-4785badf30a8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:50.668881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 66/130] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:36:50 2022] GET /identity/v3/role_assignments?user.id=af8505067983430097b469a5330dbcb3&scope.project.id=42291678aa1842ff9db31b9d1e39dc2c&role.id=5f5e6a876c4a4fb0869e5bea07e7a65c => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:50.668881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9a080-7c0f-41d7-9737-4785badf30a8 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:50.752888 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bffb0-13b0-4898-bf0d-d4281c60bd76 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:50.829946 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c524dbc-d150-4c22-8831-808ffc43b11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:50.830533 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c524dbc-d150-4c22-8831-808ffc43b11a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:50.830823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c524dbc-d150-4c22-8831-808ffc43b11a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:50.831091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c524dbc-d150-4c22-8831-808ffc43b11a None admin] PATH_INFO: `/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:50.833173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c524dbc-d150-4c22-8831-808ffc43b11a None admin] RBAC: Authorizing `identity:get_user(user_id=67faf276eb0a4b9baf8b596eb39d6bd2)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:50.834647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c524dbc-d150-4c22-8831-808ffc43b11a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:50.836352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 65/131] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:50 2022] GET /identity/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2 => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:50.837115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c524dbc-d150-4c22-8831-808ffc43b11a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:50.865434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9a080-7c0f-41d7-9737-4785badf30a8 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:50.904740 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:50.913614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c8e50-b097-45f2-9a81-39bacf1ce1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:50.914577 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c8e50-b097-45f2-9a81-39bacf1ce1ee None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:50.915077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c8e50-b097-45f2-9a81-39bacf1ce1ee None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:50.916007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c8e50-b097-45f2-9a81-39bacf1ce1ee None admin] PATH_INFO: `/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:50.917751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c8e50-b097-45f2-9a81-39bacf1ce1ee None admin] RBAC: Authorizing `identity:get_user(user_id=67faf276eb0a4b9baf8b596eb39d6bd2)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:50.919863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c8e50-b097-45f2-9a81-39bacf1ce1ee None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:50.922414 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 67/132] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:50 2022] GET /identity/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2 => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:50.923510 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c8e50-b097-45f2-9a81-39bacf1ce1ee None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:50.989305 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:50.993792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c524dbc-d150-4c22-8831-808ffc43b11a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:51.033354 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d511d8-a651-4240-8edc-6ffd7e51e272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:51.034236 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d511d8-a651-4240-8edc-6ffd7e51e272 None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:51.034615 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d511d8-a651-4240-8edc-6ffd7e51e272 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:51.035016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d511d8-a651-4240-8edc-6ffd7e51e272 None admin] PATH_INFO: `/v3/projects/f91e09770ccc4ff8813c6cfe33d2bffe/users/67faf276eb0a4b9baf8b596eb39d6bd2/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:51.037161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d511d8-a651-4240-8edc-6ffd7e51e272 None admin] RBAC: Authorizing `identity:create_grant(project_id=f91e09770ccc4ff8813c6cfe33d2bffe, user_id=67faf276eb0a4b9baf8b596eb39d6bd2, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:51.038830 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d511d8-a651-4240-8edc-6ffd7e51e272 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:51.053917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c8e50-b097-45f2-9a81-39bacf1ce1ee None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:51.055333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 66/133] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:36:50 2022] PUT /identity/v3/projects/f91e09770ccc4ff8813c6cfe33d2bffe/users/67faf276eb0a4b9baf8b596eb39d6bd2/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:36:51.056318 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d511d8-a651-4240-8edc-6ffd7e51e272 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:51.092887 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69556ca7-5e29-4153-8b8b-9602e0cf43eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:51.093594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69556ca7-5e29-4153-8b8b-9602e0cf43eb None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:51.093939 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69556ca7-5e29-4153-8b8b-9602e0cf43eb None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:51.094223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69556ca7-5e29-4153-8b8b-9602e0cf43eb None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:51.095638 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69556ca7-5e29-4153-8b8b-9602e0cf43eb None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:51.097327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69556ca7-5e29-4153-8b8b-9602e0cf43eb None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:51.099229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 68/134] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:50 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:51.100551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd568d56-23cb-4acd-b9e1-34a821192e46 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:51.100841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd568d56-23cb-4acd-b9e1-34a821192e46 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:51.101172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd568d56-23cb-4acd-b9e1-34a821192e46 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:51.102580 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 69/135] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:51.108669 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd568d56-23cb-4acd-b9e1-34a821192e46 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:51.185460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d511d8-a651-4240-8edc-6ffd7e51e272 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:51.194352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:51.232081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd568d56-23cb-4acd-b9e1-34a821192e46 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:51.265349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74af2a-a89b-4e3a-858b-c48f92ee5ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:51.266502 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74af2a-a89b-4e3a-858b-c48f92ee5ec8 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:51.267083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74af2a-a89b-4e3a-858b-c48f92ee5ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:51.267574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74af2a-a89b-4e3a-858b-c48f92ee5ec8 None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:51.270142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74af2a-a89b-4e3a-858b-c48f92ee5ec8 None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:51.272115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74af2a-a89b-4e3a-858b-c48f92ee5ec8 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:51.272413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7af610-7112-4512-acf2-9ba7d617dcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:51.273119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7af610-7112-4512-acf2-9ba7d617dcbb None admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:51.273407 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7af610-7112-4512-acf2-9ba7d617dcbb None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:51.273690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7af610-7112-4512-acf2-9ba7d617dcbb None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c/users/67faf276eb0a4b9baf8b596eb39d6bd2/roles/548e622a9ae745d99d4cd34e68c1652c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:51.275284 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 67/136] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:51 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:51.282096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9e92f-ae38-4949-b2cb-e3652eb9b153 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:51.282584 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9e92f-ae38-4949-b2cb-e3652eb9b153 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:51.283079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9e92f-ae38-4949-b2cb-e3652eb9b153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:51.287809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7af610-7112-4512-acf2-9ba7d617dcbb None admin] RBAC: Authorizing `identity:create_grant(project_id=42291678aa1842ff9db31b9d1e39dc2c, user_id=67faf276eb0a4b9baf8b596eb39d6bd2, role_id=548e622a9ae745d99d4cd34e68c1652c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:51.290162 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7af610-7112-4512-acf2-9ba7d617dcbb None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:51.312732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-53d9e92f-ae38-4949-b2cb-e3652eb9b153 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:51.314247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-53d9e92f-ae38-4949-b2cb-e3652eb9b153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:51.315771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 70/137] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:36:51 2022] PUT /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c/users/67faf276eb0a4b9baf8b596eb39d6bd2/roles/548e622a9ae745d99d4cd34e68c1652c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:36:51.317219 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7af610-7112-4512-acf2-9ba7d617dcbb None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:51.351418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:51.436843 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 68/138] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:51.460731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9e92f-ae38-4949-b2cb-e3652eb9b153 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:51.529204 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7af610-7112-4512-acf2-9ba7d617dcbb None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:51.571344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9e92f-ae38-4949-b2cb-e3652eb9b153 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:51.598303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:51.613493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccc6bd-78e8-4066-8ae8-ae0accf9d2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:51.614081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccc6bd-78e8-4066-8ae8-ae0accf9d2d9 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:51.614369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccc6bd-78e8-4066-8ae8-ae0accf9d2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:51.614646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccc6bd-78e8-4066-8ae8-ae0accf9d2d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:51.615814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccc6bd-78e8-4066-8ae8-ae0accf9d2d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:51.617289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccc6bd-78e8-4066-8ae8-ae0accf9d2d9 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:51.617871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db026fcb-7029-4ae2-b7c4-5ce925edf569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:51.618578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db026fcb-7029-4ae2-b7c4-5ce925edf569 None admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:51.619040 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 69/139] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:36:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:51.619141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db026fcb-7029-4ae2-b7c4-5ce925edf569 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:51.619211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db026fcb-7029-4ae2-b7c4-5ce925edf569 None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c/users/67faf276eb0a4b9baf8b596eb39d6bd2/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:51.621579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db026fcb-7029-4ae2-b7c4-5ce925edf569 None admin] RBAC: Authorizing `identity:create_grant(project_id=42291678aa1842ff9db31b9d1e39dc2c, user_id=67faf276eb0a4b9baf8b596eb39d6bd2, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:51.623065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db026fcb-7029-4ae2-b7c4-5ce925edf569 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:51.624860 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccc6bd-78e8-4066-8ae8-ae0accf9d2d9 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:51.645736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 71/140] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:36:51 2022] PUT /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c/users/67faf276eb0a4b9baf8b596eb39d6bd2/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:36:51.682362 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:51.753812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccc6bd-78e8-4066-8ae8-ae0accf9d2d9 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:51.793867 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472db404-6e9c-428d-b550-d6616b7ba37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:51.794513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472db404-6e9c-428d-b550-d6616b7ba37f None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:51.794799 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472db404-6e9c-428d-b550-d6616b7ba37f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:51.795077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472db404-6e9c-428d-b550-d6616b7ba37f None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:51.795783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472db404-6e9c-428d-b550-d6616b7ba37f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:51.797266 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472db404-6e9c-428d-b550-d6616b7ba37f None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:51.822805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 70/141] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:36:51 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:36:52.400444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a65f8-3daa-4790-a6ee-525fde2b7cf3 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:52.400954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a65f8-3daa-4790-a6ee-525fde2b7cf3 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:52.401027 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a65f8-3daa-4790-a6ee-525fde2b7cf3 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:52.402955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 72/142] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:52.409165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4c8a1-783f-417c-9073-e383e3633fa1 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:52.409618 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4c8a1-783f-417c-9073-e383e3633fa1 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:52.409804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4c8a1-783f-417c-9073-e383e3633fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:52.429223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-88d4c8a1-783f-417c-9073-e383e3633fa1 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:52.430430 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-88d4c8a1-783f-417c-9073-e383e3633fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:52.504022 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 71/143] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:52.528060 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a65f8-3daa-4790-a6ee-525fde2b7cf3 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:52.640215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a65f8-3daa-4790-a6ee-525fde2b7cf3 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:52.677767 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46cd36-3640-4621-950a-44c10a14b084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:52.678349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46cd36-3640-4621-950a-44c10a14b084 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:52.678635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46cd36-3640-4621-950a-44c10a14b084 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:52.678914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46cd36-3640-4621-950a-44c10a14b084 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:52.691382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46cd36-3640-4621-950a-44c10a14b084 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-ca46cd36-3640-4621-950a-44c10a14b084 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:52.696915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:52.697812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:52.699355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:52.699355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:52.699355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:52.699355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:52.699355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:52.699355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:52.699355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:52.699355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:52.699355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:52.699355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:52.699355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:36:52.699355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:36:52.699355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 73/144] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:36:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:52.704824 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4c8a1-783f-417c-9073-e383e3633fa1 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:52.728699 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:52.819012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4c8a1-783f-417c-9073-e383e3633fa1 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:52.856496 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c0998-40b6-41b2-8626-9fbcaaeb57e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:52.857208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c0998-40b6-41b2-8626-9fbcaaeb57e4 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:52.857497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c0998-40b6-41b2-8626-9fbcaaeb57e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:52.857778 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c0998-40b6-41b2-8626-9fbcaaeb57e4 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:52.858522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c0998-40b6-41b2-8626-9fbcaaeb57e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:52.858837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c0998-40b6-41b2-8626-9fbcaaeb57e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:52.860240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c0998-40b6-41b2-8626-9fbcaaeb57e4 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:52.869436 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 72/145] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:52.876371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46cd36-3640-4621-950a-44c10a14b084 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:52.929179 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:52.986988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46cd36-3640-4621-950a-44c10a14b084 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:53.023519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64176ac-b511-4274-ba06-62b9b00b0fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:53.024175 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64176ac-b511-4274-ba06-62b9b00b0fe7 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:53.024496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64176ac-b511-4274-ba06-62b9b00b0fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:53.024775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64176ac-b511-4274-ba06-62b9b00b0fe7 None admin] PATH_INFO: `/v3/users/b499f31220244d41877273ff02e31ae4` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:53.025967 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64176ac-b511-4274-ba06-62b9b00b0fe7 None admin] RBAC: Authorizing `identity:get_user(user_id=b499f31220244d41877273ff02e31ae4)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:53.027493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64176ac-b511-4274-ba06-62b9b00b0fe7 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:53.029435 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 74/146] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:52 2022] GET /identity/v3/users/b499f31220244d41877273ff02e31ae4 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:53.035036 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c0998-40b6-41b2-8626-9fbcaaeb57e4 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:53.118258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:53.161086 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c0998-40b6-41b2-8626-9fbcaaeb57e4 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:53.209900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba2a7a-3fc2-4044-a200-edb703d1f3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:53.210547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba2a7a-3fc2-4044-a200-edb703d1f3bb None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:53.210835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba2a7a-3fc2-4044-a200-edb703d1f3bb None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:53.211114 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba2a7a-3fc2-4044-a200-edb703d1f3bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:53.291802 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba2a7a-3fc2-4044-a200-edb703d1f3bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:53.293704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba2a7a-3fc2-4044-a200-edb703d1f3bb None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:53.379154 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:53.407218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 73/147] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:36:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:36:53.412965 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64176ac-b511-4274-ba06-62b9b00b0fe7 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:53.540619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64176ac-b511-4274-ba06-62b9b00b0fe7 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:53.555590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:53.578128 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed6579-3ba4-483f-9535-2441548c5666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:53.578804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed6579-3ba4-483f-9535-2441548c5666 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:53.579102 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed6579-3ba4-483f-9535-2441548c5666 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:53.579385 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed6579-3ba4-483f-9535-2441548c5666 None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:53.580694 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed6579-3ba4-483f-9535-2441548c5666 None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:53.582176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed6579-3ba4-483f-9535-2441548c5666 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:53.583875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 75/148] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:53 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:53.590545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba2a7a-3fc2-4044-a200-edb703d1f3bb None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:53.703926 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba2a7a-3fc2-4044-a200-edb703d1f3bb None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:53.742003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478bf8c3-6556-4c08-834b-015ec6f0c40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:53.742635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478bf8c3-6556-4c08-834b-015ec6f0c40e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:53.742919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478bf8c3-6556-4c08-834b-015ec6f0c40e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:53.743195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478bf8c3-6556-4c08-834b-015ec6f0c40e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:53.744009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478bf8c3-6556-4c08-834b-015ec6f0c40e None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=42291678aa1842ff9db31b9d1e39dc2c, user.id=b499f31220244d41877273ff02e31ae4) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:53.744343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478bf8c3-6556-4c08-834b-015ec6f0c40e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:53.746580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478bf8c3-6556-4c08-834b-015ec6f0c40e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:53.753230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 74/149] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:36:53 2022] GET /identity/v3/role_assignments?user.id=b499f31220244d41877273ff02e31ae4&scope.project.id=42291678aa1842ff9db31b9d1e39dc2c&role.id=19557e941d6f475ba92cf862846f9872 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:54.031924 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c6490-c515-4c4b-8884-bf23c9c17a51 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:54.032304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c6490-c515-4c4b-8884-bf23c9c17a51 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:54.032564 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c6490-c515-4c4b-8884-bf23c9c17a51 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:54.034179 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 76/150] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:54.050261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773c239-116d-458a-a0fc-5b15b13e0274 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:54.050554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773c239-116d-458a-a0fc-5b15b13e0274 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:54.050828 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773c239-116d-458a-a0fc-5b15b13e0274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:54.073348 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-6773c239-116d-458a-a0fc-5b15b13e0274 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:54.075063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-6773c239-116d-458a-a0fc-5b15b13e0274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:54.115448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:54.162432 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 75/151] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:54.186803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c6490-c515-4c4b-8884-bf23c9c17a51 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:54.295320 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2322ce-9dac-4fc7-94d4-6bd649f468db None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:54.295638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2322ce-9dac-4fc7-94d4-6bd649f468db None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:54.295892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2322ce-9dac-4fc7-94d4-6bd649f468db None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:54.297364 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 76/152] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:36:54.302706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c991a1-db82-4c9d-8b80-95d0ae6708ad None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:54.302968 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c991a1-db82-4c9d-8b80-95d0ae6708ad None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:54.303248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c991a1-db82-4c9d-8b80-95d0ae6708ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:54.309839 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c6490-c515-4c4b-8884-bf23c9c17a51 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:54.327339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-63c991a1-db82-4c9d-8b80-95d0ae6708ad None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:54.329024 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-63c991a1-db82-4c9d-8b80-95d0ae6708ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:54.366417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a2a04-f56a-4369-8a9f-ffa1bc76e891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:54.367504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a2a04-f56a-4369-8a9f-ffa1bc76e891 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:54.367976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a2a04-f56a-4369-8a9f-ffa1bc76e891 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:54.368463 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a2a04-f56a-4369-8a9f-ffa1bc76e891 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:54.393337 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a2a04-f56a-4369-8a9f-ffa1bc76e891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-c13a2a04-f56a-4369-8a9f-ffa1bc76e891 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:54.399009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:54.399848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:54.400639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:54.400639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:54.400639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:54.400639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:54.400639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:54.400639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:54.400639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:54.400639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:54.400639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:54.400639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:54.400639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:36:54.400639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:36:54.404149 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 77/153] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:54.416584 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a2a04-f56a-4369-8a9f-ffa1bc76e891 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:54.424859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 77/154] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:36:54.452485 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c991a1-db82-4c9d-8b80-95d0ae6708ad None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:54.491081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:54.571262 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:54.574902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c991a1-db82-4c9d-8b80-95d0ae6708ad None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:54.611193 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a2a04-f56a-4369-8a9f-ffa1bc76e891 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:54.621093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020a3a5-14f1-41a2-a221-c647acfdff76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:54.622395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020a3a5-14f1-41a2-a221-c647acfdff76 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:54.622895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020a3a5-14f1-41a2-a221-c647acfdff76 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:54.623367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020a3a5-14f1-41a2-a221-c647acfdff76 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:54.642139 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020a3a5-14f1-41a2-a221-c647acfdff76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-6020a3a5-14f1-41a2-a221-c647acfdff76 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:54.647504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:54.648385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:54.649387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:54.649387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:54.649387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:54.649387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:54.649387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:54.649387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:54.649387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:54.649387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:54.649387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:54.649387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:54.649387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:36:54.649387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:36:54.656396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a8b45-2079-4004-852e-a72fe37cc1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:54.656396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a8b45-2079-4004-852e-a72fe37cc1a7 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:54.656396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a8b45-2079-4004-852e-a72fe37cc1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:54.656396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a8b45-2079-4004-852e-a72fe37cc1a7 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:54.656773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a8b45-2079-4004-852e-a72fe37cc1a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:54.656773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a8b45-2079-4004-852e-a72fe37cc1a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:54.657089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 78/155] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:36:54.658665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a8b45-2079-4004-852e-a72fe37cc1a7 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:54.659256 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020a3a5-14f1-41a2-a221-c647acfdff76 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:54.668933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 78/156] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:36:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:54.674670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a8b45-2079-4004-852e-a72fe37cc1a7 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:54.783583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020a3a5-14f1-41a2-a221-c647acfdff76 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:54.836757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826958ac-67ea-49e1-8108-2c5a387ab06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:54.837718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826958ac-67ea-49e1-8108-2c5a387ab06b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:54.838177 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826958ac-67ea-49e1-8108-2c5a387ab06b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:54.838648 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826958ac-67ea-49e1-8108-2c5a387ab06b None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:54.839661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826958ac-67ea-49e1-8108-2c5a387ab06b None admin] RBAC: Adding query filter params (name=member) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:54.840352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826958ac-67ea-49e1-8108-2c5a387ab06b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:54.842225 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826958ac-67ea-49e1-8108-2c5a387ab06b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:54.842779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a8b45-2079-4004-852e-a72fe37cc1a7 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:54.849741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:54.854631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 79/157] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:36:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:54.858866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826958ac-67ea-49e1-8108-2c5a387ab06b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:54.888792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:54.895085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48f74c-5703-4312-9674-28e281f25e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:54.895667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48f74c-5703-4312-9674-28e281f25e8e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:54.895951 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48f74c-5703-4312-9674-28e281f25e8e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:54.896225 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48f74c-5703-4312-9674-28e281f25e8e None admin] PATH_INFO: `/v3/users/af8505067983430097b469a5330dbcb3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:54.897418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48f74c-5703-4312-9674-28e281f25e8e None admin] RBAC: Authorizing `identity:get_user(user_id=af8505067983430097b469a5330dbcb3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:54.898876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48f74c-5703-4312-9674-28e281f25e8e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:54.900804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 79/158] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:54 2022] GET /identity/v3/users/af8505067983430097b469a5330dbcb3 => generated 289 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:54.906895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48f74c-5703-4312-9674-28e281f25e8e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:55.080570 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48f74c-5703-4312-9674-28e281f25e8e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:55.109305 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826958ac-67ea-49e1-8108-2c5a387ab06b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:55.117096 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f5a76-b861-4795-ba0f-6b3ebc7d28b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:55.117751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f5a76-b861-4795-ba0f-6b3ebc7d28b4 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:55.118026 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f5a76-b861-4795-ba0f-6b3ebc7d28b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:55.118340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f5a76-b861-4795-ba0f-6b3ebc7d28b4 None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:55.119557 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f5a76-b861-4795-ba0f-6b3ebc7d28b4 None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:55.121922 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f5a76-b861-4795-ba0f-6b3ebc7d28b4 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:55.124029 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 80/159] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:54 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:55.131360 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f5a76-b861-4795-ba0f-6b3ebc7d28b4 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:55.195927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:55.209003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8f384-8b43-463c-bf80-268ac29a514a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:55.209675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8f384-8b43-463c-bf80-268ac29a514a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:55.209959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8f384-8b43-463c-bf80-268ac29a514a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:55.210233 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8f384-8b43-463c-bf80-268ac29a514a None admin] PATH_INFO: `/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:55.211441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8f384-8b43-463c-bf80-268ac29a514a None admin] RBAC: Authorizing `identity:get_user(user_id=67faf276eb0a4b9baf8b596eb39d6bd2)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:55.212917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8f384-8b43-463c-bf80-268ac29a514a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:55.215635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 80/160] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:54 2022] GET /identity/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2 => generated 275 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:55.221285 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8f384-8b43-463c-bf80-268ac29a514a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:55.286132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f5a76-b861-4795-ba0f-6b3ebc7d28b4 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:55.322693 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a5041-1b74-40d3-8c70-f4aeae141477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:55.324088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a5041-1b74-40d3-8c70-f4aeae141477 None admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:55.324604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a5041-1b74-40d3-8c70-f4aeae141477 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:55.325110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a5041-1b74-40d3-8c70-f4aeae141477 None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c/users/af8505067983430097b469a5330dbcb3/roles/5f5e6a876c4a4fb0869e5bea07e7a65c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:55.327326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a5041-1b74-40d3-8c70-f4aeae141477 None admin] RBAC: Authorizing `identity:create_grant(project_id=42291678aa1842ff9db31b9d1e39dc2c, user_id=af8505067983430097b469a5330dbcb3, role_id=5f5e6a876c4a4fb0869e5bea07e7a65c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:55.329040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a5041-1b74-40d3-8c70-f4aeae141477 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:55.350251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 81/161] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:36:55 2022] PUT /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c/users/af8505067983430097b469a5330dbcb3/roles/5f5e6a876c4a4fb0869e5bea07e7a65c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:36:55.399828 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:55.415873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8f384-8b43-463c-bf80-268ac29a514a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:55.462202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccb9ba-e50c-46a7-bb22-d6e6c95e301d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:55.463216 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccb9ba-e50c-46a7-bb22-d6e6c95e301d None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:55.463689 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccb9ba-e50c-46a7-bb22-d6e6c95e301d None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:55.464320 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccb9ba-e50c-46a7-bb22-d6e6c95e301d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:55.553011 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ccb9ba-e50c-46a7-bb22-d6e6c95e301d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:55.554819 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ccb9ba-e50c-46a7-bb22-d6e6c95e301d None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:55.582346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bb57e-2db7-4b39-87c3-d32d17b604e5 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:55.583298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bb57e-2db7-4b39-87c3-d32d17b604e5 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:55.583801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bb57e-2db7-4b39-87c3-d32d17b604e5 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:55.585477 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 82/162] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:36:55.592014 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776038b4-a131-4d48-8c4d-4ac966d77bca None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:55.592634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776038b4-a131-4d48-8c4d-4ac966d77bca None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:55.593101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776038b4-a131-4d48-8c4d-4ac966d77bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:55.630133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-776038b4-a131-4d48-8c4d-4ac966d77bca None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:55.633320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-776038b4-a131-4d48-8c4d-4ac966d77bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:55.748082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:55.780262 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 81/163] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:36:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:36:55.780976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 83/164] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:36:55.791281 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccb9ba-e50c-46a7-bb22-d6e6c95e301d None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:55.829801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776038b4-a131-4d48-8c4d-4ac966d77bca None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:55.880721 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:55.914261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccb9ba-e50c-46a7-bb22-d6e6c95e301d None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:55.970408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1532fea-8185-4f2e-9088-e238cbac3ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:55.971076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1532fea-8185-4f2e-9088-e238cbac3ef0 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:55.971357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1532fea-8185-4f2e-9088-e238cbac3ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:55.971635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1532fea-8185-4f2e-9088-e238cbac3ef0 None admin] PATH_INFO: `/v3/projects/f91e09770ccc4ff8813c6cfe33d2bffe` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:55.976523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1532fea-8185-4f2e-9088-e238cbac3ef0 None admin] RBAC: Authorizing `identity:get_project(project_id=f91e09770ccc4ff8813c6cfe33d2bffe)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:55.978037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1532fea-8185-4f2e-9088-e238cbac3ef0 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:55.979900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 82/165] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:55 2022] GET /identity/v3/projects/f91e09770ccc4ff8813c6cfe33d2bffe => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:55.981123 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b82a9-ac1b-41fe-a502-e7e687a8603b None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:55.981410 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b82a9-ac1b-41fe-a502-e7e687a8603b None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:55.981782 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b82a9-ac1b-41fe-a502-e7e687a8603b None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:55.983424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 83/166] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:55.986735 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b82a9-ac1b-41fe-a502-e7e687a8603b None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:56.006444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776038b4-a131-4d48-8c4d-4ac966d77bca None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:56.066776 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:56.085028 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d0dd1-eb55-4061-92c5-cf2a5e28eba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:56.085704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d0dd1-eb55-4061-92c5-cf2a5e28eba0 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:56.085995 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d0dd1-eb55-4061-92c5-cf2a5e28eba0 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:56.086276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d0dd1-eb55-4061-92c5-cf2a5e28eba0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:56.102787 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d0dd1-eb55-4061-92c5-cf2a5e28eba0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:56.116184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b82a9-ac1b-41fe-a502-e7e687a8603b None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-a72d0dd1-eb55-4061-92c5-cf2a5e28eba0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:56.120900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:56.121863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:56.123222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:56.123222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:56.123222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:56.123222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:56.123222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:56.123222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:56.123222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:56.123222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:56.123222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:56.123222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:56.123222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:36:56.123222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:36:56.123222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 84/167] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:56.123863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7460ae1-1509-4df5-acce-88b96552a836 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:56.124451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7460ae1-1509-4df5-acce-88b96552a836 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:56.124451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7460ae1-1509-4df5-acce-88b96552a836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:56.156352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3dd3e7-2a63-4b3b-9afb-e80b25e64b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:56.156914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3dd3e7-2a63-4b3b-9afb-e80b25e64b6b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:56.157200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3dd3e7-2a63-4b3b-9afb-e80b25e64b6b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:56.157585 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3dd3e7-2a63-4b3b-9afb-e80b25e64b6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:56.158486 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3dd3e7-2a63-4b3b-9afb-e80b25e64b6b None admin] RBAC: Adding query filter params (role.id=f3b7ffe4d1f04af18f7da8d9a51031b3, scope.project.id=f91e09770ccc4ff8813c6cfe33d2bffe, user.id=67faf276eb0a4b9baf8b596eb39d6bd2) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:56.158858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3dd3e7-2a63-4b3b-9afb-e80b25e64b6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:56.160524 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3dd3e7-2a63-4b3b-9afb-e80b25e64b6b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:56.167299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 84/168] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:36:55 2022] GET /identity/v3/role_assignments?user.id=67faf276eb0a4b9baf8b596eb39d6bd2&scope.project.id=f91e09770ccc4ff8813c6cfe33d2bffe&role.id=f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:56.168264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3dd3e7-2a63-4b3b-9afb-e80b25e64b6b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:56.169314 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d7460ae1-1509-4df5-acce-88b96552a836 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:56.170540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d7460ae1-1509-4df5-acce-88b96552a836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:56.264454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 85/169] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:56.265768 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ea23a-f335-408d-9e41-f54fb31b7c34 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:56.266065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ea23a-f335-408d-9e41-f54fb31b7c34 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:56.266346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ea23a-f335-408d-9e41-f54fb31b7c34 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:56.273417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 86/170] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:56.279744 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0e7fa-388f-4646-9d14-4117c63c0ffb None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:56.280048 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0e7fa-388f-4646-9d14-4117c63c0ffb None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:56.280332 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0e7fa-388f-4646-9d14-4117c63c0ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:56.313798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-94a0e7fa-388f-4646-9d14-4117c63c0ffb None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:56.315016 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-94a0e7fa-388f-4646-9d14-4117c63c0ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:56.348655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:56.375111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3dd3e7-2a63-4b3b-9afb-e80b25e64b6b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:56.414016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:56.429917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a773bbe-1e59-4652-a825-7c88383641d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:56.430556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a773bbe-1e59-4652-a825-7c88383641d1 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:56.430846 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a773bbe-1e59-4652-a825-7c88383641d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:56.431284 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a773bbe-1e59-4652-a825-7c88383641d1 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:56.432040 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a773bbe-1e59-4652-a825-7c88383641d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:56.432382 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a773bbe-1e59-4652-a825-7c88383641d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:56.434518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a773bbe-1e59-4652-a825-7c88383641d1 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:56.444507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 85/171] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:36:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:56.445591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a773bbe-1e59-4652-a825-7c88383641d1 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:56.463256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 87/172] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:36:56.465390 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0e7fa-388f-4646-9d14-4117c63c0ffb None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:56.560834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a773bbe-1e59-4652-a825-7c88383641d1 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:56.597822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557c884-ac41-4025-a768-f1b01144569e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:56.598626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557c884-ac41-4025-a768-f1b01144569e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:56.598975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557c884-ac41-4025-a768-f1b01144569e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:56.599365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557c884-ac41-4025-a768-f1b01144569e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:56.611312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557c884-ac41-4025-a768-f1b01144569e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-5557c884-ac41-4025-a768-f1b01144569e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:56.616949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:56.617806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:56.618630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:56.618630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:56.618630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:56.618630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:56.618630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:56.618630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:56.618630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:56.618630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:56.618630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:56.618630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:56.618630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:36:56.618630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:36:56.618630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 86/173] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:36:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:56.619275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557c884-ac41-4025-a768-f1b01144569e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:56.669539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:56.734758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557c884-ac41-4025-a768-f1b01144569e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:56.770268 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0e7fa-388f-4646-9d14-4117c63c0ffb None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:56.774672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efda54b-5f63-4f7c-9371-9be49d345c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:56.775327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efda54b-5f63-4f7c-9371-9be49d345c43 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:56.775611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efda54b-5f63-4f7c-9371-9be49d345c43 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:56.775891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efda54b-5f63-4f7c-9371-9be49d345c43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:56.788548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efda54b-5f63-4f7c-9371-9be49d345c43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-0efda54b-5f63-4f7c-9371-9be49d345c43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:56.795807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:56.796762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:56.797536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:56.797536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:56.797536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:56.797536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:56.797536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:56.797536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:56.797536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:56.797536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:56.797536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:56.797536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:56.797536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:36:56.797536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:36:56.799392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 87/174] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:36:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:56.800807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efda54b-5f63-4f7c-9371-9be49d345c43 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:56.864628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfcf98c-6f75-4604-ad8f-ca9be85d0c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:56.865371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcf98c-6f75-4604-ad8f-ca9be85d0c12 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:56.865656 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcf98c-6f75-4604-ad8f-ca9be85d0c12 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:56.865932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcf98c-6f75-4604-ad8f-ca9be85d0c12 None admin] PATH_INFO: `/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:56.868999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfcf98c-6f75-4604-ad8f-ca9be85d0c12 None admin] RBAC: Authorizing `identity:get_user(user_id=67faf276eb0a4b9baf8b596eb39d6bd2)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:56.871474 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfcf98c-6f75-4604-ad8f-ca9be85d0c12 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:56.872970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 88/175] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:56 2022] GET /identity/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2 => generated 275 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:56.880361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfcf98c-6f75-4604-ad8f-ca9be85d0c12 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:56.934409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:56.988949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efda54b-5f63-4f7c-9371-9be49d345c43 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:56.990458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:57.028890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165877e6-4c41-45b0-877c-ab16139a6730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:57.029764 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165877e6-4c41-45b0-877c-ab16139a6730 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:57.030049 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165877e6-4c41-45b0-877c-ab16139a6730 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:57.030322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165877e6-4c41-45b0-877c-ab16139a6730 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:57.031195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165877e6-4c41-45b0-877c-ab16139a6730 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:57.031503 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165877e6-4c41-45b0-877c-ab16139a6730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:57.033054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165877e6-4c41-45b0-877c-ab16139a6730 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:57.042332 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 88/176] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:36:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:57.043144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165877e6-4c41-45b0-877c-ab16139a6730 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:57.097123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfcf98c-6f75-4604-ad8f-ca9be85d0c12 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:57.138753 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f16b8-c19e-43e6-b4ad-4e6706a6668e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:57.139726 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f16b8-c19e-43e6-b4ad-4e6706a6668e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:57.140028 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f16b8-c19e-43e6-b4ad-4e6706a6668e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:57.140340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f16b8-c19e-43e6-b4ad-4e6706a6668e None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:57.141392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f16b8-c19e-43e6-b4ad-4e6706a6668e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:57.141715 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f16b8-c19e-43e6-b4ad-4e6706a6668e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:57.143411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f16b8-c19e-43e6-b4ad-4e6706a6668e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:57.154739 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 89/177] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:57.155667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f16b8-c19e-43e6-b4ad-4e6706a6668e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:57.166562 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165877e6-4c41-45b0-877c-ab16139a6730 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:57.211410 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c190c0e-9d15-4a85-9f35-1a94c4b2bae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:57.212305 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c190c0e-9d15-4a85-9f35-1a94c4b2bae9 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:57.213016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c190c0e-9d15-4a85-9f35-1a94c4b2bae9 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:57.213370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c190c0e-9d15-4a85-9f35-1a94c4b2bae9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:57.257682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:57.257957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:57.282129 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f16b8-c19e-43e6-b4ad-4e6706a6668e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:57.301802 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c190c0e-9d15-4a85-9f35-1a94c4b2bae9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:57.303273 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c190c0e-9d15-4a85-9f35-1a94c4b2bae9 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:57.318146 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65200e93-7629-466f-8eae-f6bbbab2e8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:57.319022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65200e93-7629-466f-8eae-f6bbbab2e8a9 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:57.319141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65200e93-7629-466f-8eae-f6bbbab2e8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:57.319327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65200e93-7629-466f-8eae-f6bbbab2e8a9 None admin] PATH_INFO: `/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:57.320520 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65200e93-7629-466f-8eae-f6bbbab2e8a9 None admin] RBAC: Authorizing `identity:get_user(user_id=67faf276eb0a4b9baf8b596eb39d6bd2)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:57.321969 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65200e93-7629-466f-8eae-f6bbbab2e8a9 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:57.323924 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 90/178] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:57 2022] GET /identity/v3/users/67faf276eb0a4b9baf8b596eb39d6bd2 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:57.324857 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380d39f-366e-4550-b31c-44943570da2a None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:57.325141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380d39f-366e-4550-b31c-44943570da2a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:57.325414 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380d39f-366e-4550-b31c-44943570da2a None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:57.326636 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 91/179] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:57.327332 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380d39f-366e-4550-b31c-44943570da2a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:57.419461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 89/180] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:36:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:36:57.420894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744024a1-bbcc-4f5b-977d-726076a10d55 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:57.421184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744024a1-bbcc-4f5b-977d-726076a10d55 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:57.421462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744024a1-bbcc-4f5b-977d-726076a10d55 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:57.422850 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 90/181] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:57.427305 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744024a1-bbcc-4f5b-977d-726076a10d55 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:57.457780 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380d39f-366e-4550-b31c-44943570da2a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:57.496576 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058b441-90a8-410c-b9a0-6186cabbe71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:57.497156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058b441-90a8-410c-b9a0-6186cabbe71c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:57.497440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058b441-90a8-410c-b9a0-6186cabbe71c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:57.497723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058b441-90a8-410c-b9a0-6186cabbe71c None admin] PATH_INFO: `/v3/users/a4bf2d9337214f06a8c625d58661fb71` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:57.506426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:57.512657 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058b441-90a8-410c-b9a0-6186cabbe71c None admin] RBAC: Authorizing `identity:get_user(user_id=a4bf2d9337214f06a8c625d58661fb71)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:57.514546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058b441-90a8-410c-b9a0-6186cabbe71c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:57.520850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 92/182] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:57 2022] GET /identity/v3/users/a4bf2d9337214f06a8c625d58661fb71 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:57.520850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf342a4-8d88-46cf-8eb4-6b93b1d0e497 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:57.520850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf342a4-8d88-46cf-8eb4-6b93b1d0e497 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:57.520850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf342a4-8d88-46cf-8eb4-6b93b1d0e497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:57.532822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:57.539592 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-edf342a4-8d88-46cf-8eb4-6b93b1d0e497 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:57.540847 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-edf342a4-8d88-46cf-8eb4-6b93b1d0e497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:57.559743 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744024a1-bbcc-4f5b-977d-726076a10d55 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:57.598412 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e42af-7674-4b59-8fd4-59872c8331f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:57.599042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e42af-7674-4b59-8fd4-59872c8331f4 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:57.599326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e42af-7674-4b59-8fd4-59872c8331f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:57.599603 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e42af-7674-4b59-8fd4-59872c8331f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:57.615772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 93/183] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:57.616544 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf342a4-8d88-46cf-8eb4-6b93b1d0e497 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:57.688370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e42af-7674-4b59-8fd4-59872c8331f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:57.689831 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e42af-7674-4b59-8fd4-59872c8331f4 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:57.721157 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf342a4-8d88-46cf-8eb4-6b93b1d0e497 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:57.754150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:57.760347 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd2daa-2d98-421f-a67f-f02d9aa0c2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:57.761648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd2daa-2d98-421f-a67f-f02d9aa0c2a5 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:57.761930 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd2daa-2d98-421f-a67f-f02d9aa0c2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:57.762350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd2daa-2d98-421f-a67f-f02d9aa0c2a5 None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:57.763683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd2daa-2d98-421f-a67f-f02d9aa0c2a5 None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:57.765309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd2daa-2d98-421f-a67f-f02d9aa0c2a5 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:57.767175 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 94/184] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:57 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:57.768487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6073d5-c451-46b7-aac4-bca75a06ace8 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:57.768900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6073d5-c451-46b7-aac4-bca75a06ace8 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:57.769294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6073d5-c451-46b7-aac4-bca75a06ace8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:57.777470 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:57.788944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-8d6073d5-c451-46b7-aac4-bca75a06ace8 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:57.790424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8d6073d5-c451-46b7-aac4-bca75a06ace8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:57.803547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 91/185] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:36:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:36:57.804481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e42af-7674-4b59-8fd4-59872c8331f4 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:57.877556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 95/186] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:57.885667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6073d5-c451-46b7-aac4-bca75a06ace8 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:57.912311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e42af-7674-4b59-8fd4-59872c8331f4 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:57.957259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194c9ae-0e49-40c4-b38f-61fe784a4684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:57.957986 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194c9ae-0e49-40c4-b38f-61fe784a4684 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:57.958278 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194c9ae-0e49-40c4-b38f-61fe784a4684 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:57.958561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194c9ae-0e49-40c4-b38f-61fe784a4684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:57.992411 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:58.007460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6073d5-c451-46b7-aac4-bca75a06ace8 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:58.044485 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194c9ae-0e49-40c4-b38f-61fe784a4684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:58.045970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194c9ae-0e49-40c4-b38f-61fe784a4684 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:58.047657 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f84330-f2d3-4412-b098-991091122bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:58.048333 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f84330-f2d3-4412-b098-991091122bb3 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:58.048623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f84330-f2d3-4412-b098-991091122bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:58.048903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f84330-f2d3-4412-b098-991091122bb3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:58.050804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:58.061968 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f84330-f2d3-4412-b098-991091122bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-97f84330-f2d3-4412-b098-991091122bb3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:58.067068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:58.067923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:58.069090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:58.069090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:58.069090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:58.069090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:58.069090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:58.069090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:58.069090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:58.069090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:58.069090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:58.069090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:58.069090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:36:58.069090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:36:58.069090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 96/187] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:36:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:58.073758 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f84330-f2d3-4412-b098-991091122bb3 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:58.160201 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 92/188] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:36:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:36:58.165783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194c9ae-0e49-40c4-b38f-61fe784a4684 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:58.182241 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f84330-f2d3-4412-b098-991091122bb3 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:58.220647 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf8058-8f44-414d-87a7-258ebee085ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:58.221503 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf8058-8f44-414d-87a7-258ebee085ec None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:58.221859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf8058-8f44-414d-87a7-258ebee085ec None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:58.222212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf8058-8f44-414d-87a7-258ebee085ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:58.223029 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf8058-8f44-414d-87a7-258ebee085ec None admin] RBAC: Adding query filter params (role.id=f3b7ffe4d1f04af18f7da8d9a51031b3, scope.project.id=42291678aa1842ff9db31b9d1e39dc2c, user.id=67faf276eb0a4b9baf8b596eb39d6bd2) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:58.223344 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf8058-8f44-414d-87a7-258ebee085ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:58.224862 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf8058-8f44-414d-87a7-258ebee085ec None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:58.231199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 97/189] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:36:58 2022] GET /identity/v3/role_assignments?user.id=67faf276eb0a4b9baf8b596eb39d6bd2&scope.project.id=42291678aa1842ff9db31b9d1e39dc2c&role.id=f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:58.232055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf8058-8f44-414d-87a7-258ebee085ec None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:58.250809 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:58.303853 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194c9ae-0e49-40c4-b38f-61fe784a4684 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:58.319306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:58.358198 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c804ba-8356-4e2b-9e63-ee067bc8c28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:58.358897 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c804ba-8356-4e2b-9e63-ee067bc8c28d None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:58.359186 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c804ba-8356-4e2b-9e63-ee067bc8c28d None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:58.359667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c804ba-8356-4e2b-9e63-ee067bc8c28d None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:58.361871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c804ba-8356-4e2b-9e63-ee067bc8c28d None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:58.364023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c804ba-8356-4e2b-9e63-ee067bc8c28d None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:58.371709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 93/190] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:58 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:58.371709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c804ba-8356-4e2b-9e63-ee067bc8c28d None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:58.371962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf8058-8f44-414d-87a7-258ebee085ec None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:58.413873 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbac64-28fe-4e5c-9e2c-7bb484671d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:58.414533 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbac64-28fe-4e5c-9e2c-7bb484671d5e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:58.414811 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbac64-28fe-4e5c-9e2c-7bb484671d5e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:58.415096 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbac64-28fe-4e5c-9e2c-7bb484671d5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:58.416317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dbac64-28fe-4e5c-9e2c-7bb484671d5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:58.417848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dbac64-28fe-4e5c-9e2c-7bb484671d5e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:58.419642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 98/191] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:36:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:58.420725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbac64-28fe-4e5c-9e2c-7bb484671d5e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:58.519896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c804ba-8356-4e2b-9e63-ee067bc8c28d None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:58.544591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbac64-28fe-4e5c-9e2c-7bb484671d5e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:58.551694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:58.557247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe619f-c518-4652-9d03-e2e432993acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:58.558600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe619f-c518-4652-9d03-e2e432993acb None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:58.558727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe619f-c518-4652-9d03-e2e432993acb None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:58.558979 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe619f-c518-4652-9d03-e2e432993acb None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:58.559708 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe619f-c518-4652-9d03-e2e432993acb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:58.560009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe619f-c518-4652-9d03-e2e432993acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:58.561583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe619f-c518-4652-9d03-e2e432993acb None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:58.570150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 94/192] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:36:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:58.570150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70e4bc-19ad-4ae1-863e-3266d146f166 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:58.570150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70e4bc-19ad-4ae1-863e-3266d146f166 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:58.570150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70e4bc-19ad-4ae1-863e-3266d146f166 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:58.570150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 95/193] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:36:58.570150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70e4bc-19ad-4ae1-863e-3266d146f166 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:58.577330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:58.586010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ab42b-744a-4cd3-b0d4-bff7cc56be75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:58.586660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ab42b-744a-4cd3-b0d4-bff7cc56be75 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:58.586945 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ab42b-744a-4cd3-b0d4-bff7cc56be75 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:58.587225 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ab42b-744a-4cd3-b0d4-bff7cc56be75 None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:58.588419 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ab42b-744a-4cd3-b0d4-bff7cc56be75 None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:58.589919 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ab42b-744a-4cd3-b0d4-bff7cc56be75 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:58.591592 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 99/194] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:58 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:58.592352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ab42b-744a-4cd3-b0d4-bff7cc56be75 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:58.683237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70e4bc-19ad-4ae1-863e-3266d146f166 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:58.693264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ab42b-744a-4cd3-b0d4-bff7cc56be75 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:58.718883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b1a49-a1bb-4774-bec9-11b41129f11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:58.719501 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b1a49-a1bb-4774-bec9-11b41129f11d None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:58.719773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b1a49-a1bb-4774-bec9-11b41129f11d None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:58.720049 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b1a49-a1bb-4774-bec9-11b41129f11d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:58.720884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b1a49-a1bb-4774-bec9-11b41129f11d None admin] RBAC: Adding query filter params (role.id=548e622a9ae745d99d4cd34e68c1652c, scope.project.id=42291678aa1842ff9db31b9d1e39dc2c, user.id=67faf276eb0a4b9baf8b596eb39d6bd2) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:58.721189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b1a49-a1bb-4774-bec9-11b41129f11d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:58.722639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b1a49-a1bb-4774-bec9-11b41129f11d None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:58.727528 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca2fb8-677b-4db2-a67d-2ede35db3470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:58.728143 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca2fb8-677b-4db2-a67d-2ede35db3470 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:58.728603 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 96/195] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:36:58 2022] GET /identity/v3/role_assignments?user.id=67faf276eb0a4b9baf8b596eb39d6bd2&scope.project.id=42291678aa1842ff9db31b9d1e39dc2c&role.id=548e622a9ae745d99d4cd34e68c1652c => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:58.728725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca2fb8-677b-4db2-a67d-2ede35db3470 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:58.728725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca2fb8-677b-4db2-a67d-2ede35db3470 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:58.729364 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b1a49-a1bb-4774-bec9-11b41129f11d None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:58.729570 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca2fb8-677b-4db2-a67d-2ede35db3470 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:58.735878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca2fb8-677b-4db2-a67d-2ede35db3470 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:58.767145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 100/196] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:36:58 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:36:58.768521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c6b28-ebbf-4229-b14c-1a36582c5ef0 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:58.768814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c6b28-ebbf-4229-b14c-1a36582c5ef0 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:58.769088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c6b28-ebbf-4229-b14c-1a36582c5ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:58.790582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-331c6b28-ebbf-4229-b14c-1a36582c5ef0 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:36:58.792489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-331c6b28-ebbf-4229-b14c-1a36582c5ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:36:58.814394 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:58.854868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:58.895554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b1a49-a1bb-4774-bec9-11b41129f11d None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:58.896557 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 101/197] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:36:58.897897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c6b28-ebbf-4229-b14c-1a36582c5ef0 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:58.957464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21edece-badd-47d4-9a7b-dc718535e243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:58.958450 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21edece-badd-47d4-9a7b-dc718535e243 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:58.958928 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21edece-badd-47d4-9a7b-dc718535e243 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:58.959359 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21edece-badd-47d4-9a7b-dc718535e243 None admin] PATH_INFO: `/v3/users/a4bf2d9337214f06a8c625d58661fb71` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:58.961002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21edece-badd-47d4-9a7b-dc718535e243 None admin] RBAC: Authorizing `identity:get_user(user_id=a4bf2d9337214f06a8c625d58661fb71)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:58.962527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21edece-badd-47d4-9a7b-dc718535e243 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:58.965548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 97/198] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:58 2022] GET /identity/v3/users/a4bf2d9337214f06a8c625d58661fb71 => generated 283 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:58.966502 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21edece-badd-47d4-9a7b-dc718535e243 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:59.051347 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c6b28-ebbf-4229-b14c-1a36582c5ef0 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:59.099845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efdcee-e5d6-410d-bfa1-8780f80ed807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:59.100830 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efdcee-e5d6-410d-bfa1-8780f80ed807 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:59.101258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efdcee-e5d6-410d-bfa1-8780f80ed807 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:59.101676 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efdcee-e5d6-410d-bfa1-8780f80ed807 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:59.102643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efdcee-e5d6-410d-bfa1-8780f80ed807 None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=15984787df4448da8a1beaa8b7b15320, user.id=a4bf2d9337214f06a8c625d58661fb71) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:59.103102 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efdcee-e5d6-410d-bfa1-8780f80ed807 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:59.103470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21edece-badd-47d4-9a7b-dc718535e243 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:59.104999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efdcee-e5d6-410d-bfa1-8780f80ed807 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:59.111151 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 102/199] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:36:58 2022] GET /identity/v3/role_assignments?user.id=a4bf2d9337214f06a8c625d58661fb71&scope.project.id=15984787df4448da8a1beaa8b7b15320&role.id=19557e941d6f475ba92cf862846f9872 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:59.112234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efdcee-e5d6-410d-bfa1-8780f80ed807 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:59.142753 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:59.160682 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb57b1-be40-4d69-893f-38f50e8cec27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:59.161355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb57b1-be40-4d69-893f-38f50e8cec27 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:59.161639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb57b1-be40-4d69-893f-38f50e8cec27 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:59.161975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb57b1-be40-4d69-893f-38f50e8cec27 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:59.177021 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:59.178361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb57b1-be40-4d69-893f-38f50e8cec27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-28bb57b1-be40-4d69-893f-38f50e8cec27 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:36:59.187093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:36:59.188039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:36:59.188905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:36:59.188905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:36:59.188905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:36:59.188905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:36:59.188905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:36:59.188905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:36:59.188905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:36:59.188905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:36:59.188905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:36:59.188905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:36:59.188905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:36:59.188905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:36:59.188905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 98/200] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:36:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:36:59.195636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb57b1-be40-4d69-893f-38f50e8cec27 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:59.249991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efdcee-e5d6-410d-bfa1-8780f80ed807 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:59.301343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcd3b8-3f28-4681-999a-76fa73193f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:59.302171 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcd3b8-3f28-4681-999a-76fa73193f44 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:59.302649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcd3b8-3f28-4681-999a-76fa73193f44 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:59.303027 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcd3b8-3f28-4681-999a-76fa73193f44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:59.341381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb57b1-be40-4d69-893f-38f50e8cec27 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:59.386755 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609dcf8-d7f4-4986-8974-dbf6c9c8bb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:59.387767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609dcf8-d7f4-4986-8974-dbf6c9c8bb05 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:59.388348 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609dcf8-d7f4-4986-8974-dbf6c9c8bb05 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:59.388795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609dcf8-d7f4-4986-8974-dbf6c9c8bb05 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:59.389707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609dcf8-d7f4-4986-8974-dbf6c9c8bb05 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:59.390209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609dcf8-d7f4-4986-8974-dbf6c9c8bb05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:59.390856 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcd3b8-3f28-4681-999a-76fa73193f44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:59.391978 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609dcf8-d7f4-4986-8974-dbf6c9c8bb05 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:59.392545 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcd3b8-3f28-4681-999a-76fa73193f44 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:59.402622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 99/201] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:36:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:36:59.407998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609dcf8-d7f4-4986-8974-dbf6c9c8bb05 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:59.445302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:59.514866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:59.537911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 103/202] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:36:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:36:59.549430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcd3b8-3f28-4681-999a-76fa73193f44 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:59.583897 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609dcf8-d7f4-4986-8974-dbf6c9c8bb05 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:59.622981 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e56381-e75e-4bf9-95ab-1d4cc3b3a38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:59.623979 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e56381-e75e-4bf9-95ab-1d4cc3b3a38b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:59.624432 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e56381-e75e-4bf9-95ab-1d4cc3b3a38b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:59.624937 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e56381-e75e-4bf9-95ab-1d4cc3b3a38b None admin] PATH_INFO: `/v3/users/af8505067983430097b469a5330dbcb3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:59.626341 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e56381-e75e-4bf9-95ab-1d4cc3b3a38b None admin] RBAC: Authorizing `identity:get_user(user_id=af8505067983430097b469a5330dbcb3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:59.628129 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e56381-e75e-4bf9-95ab-1d4cc3b3a38b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:59.630159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 100/203] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:36:59 2022] GET /identity/v3/users/af8505067983430097b469a5330dbcb3 => generated 289 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:59.641572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e56381-e75e-4bf9-95ab-1d4cc3b3a38b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:59.674041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcd3b8-3f28-4681-999a-76fa73193f44 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:59.721368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1b4fd-e97d-44c5-aa13-ece797f1ab83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:59.722015 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1b4fd-e97d-44c5-aa13-ece797f1ab83 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:59.722300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1b4fd-e97d-44c5-aa13-ece797f1ab83 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:59.722580 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1b4fd-e97d-44c5-aa13-ece797f1ab83 None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:59.725239 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1b4fd-e97d-44c5-aa13-ece797f1ab83 None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:59.726703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1b4fd-e97d-44c5-aa13-ece797f1ab83 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:59.731027 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 104/204] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:36:59 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:59.736965 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1b4fd-e97d-44c5-aa13-ece797f1ab83 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:36:59.771529 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:59.850187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e56381-e75e-4bf9-95ab-1d4cc3b3a38b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:59.887262 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1b4fd-e97d-44c5-aa13-ece797f1ab83 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:36:59.896685 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:36:59.925173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4fc37-c5c1-4eb7-9773-d8dcbd8e560b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:59.925809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4fc37-c5c1-4eb7-9773-d8dcbd8e560b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:59.926104 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4fc37-c5c1-4eb7-9773-d8dcbd8e560b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:59.926383 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4fc37-c5c1-4eb7-9773-d8dcbd8e560b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:36:59.927194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4fc37-c5c1-4eb7-9773-d8dcbd8e560b None admin] RBAC: Adding query filter params (role.id=548e622a9ae745d99d4cd34e68c1652c, scope.project.id=15984787df4448da8a1beaa8b7b15320, user.id=a4bf2d9337214f06a8c625d58661fb71) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:36:59.927505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4fc37-c5c1-4eb7-9773-d8dcbd8e560b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:36:59.928980 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4fc37-c5c1-4eb7-9773-d8dcbd8e560b None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:36:59.938431 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 105/205] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:36:59 2022] GET /identity/v3/role_assignments?user.id=a4bf2d9337214f06a8c625d58661fb71&scope.project.id=15984787df4448da8a1beaa8b7b15320&role.id=548e622a9ae745d99d4cd34e68c1652c => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:36:59.989141 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02262a5-33b4-4d2d-a0a6-93dad6d51d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:36:59.989807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02262a5-33b4-4d2d-a0a6-93dad6d51d4b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:36:59.990091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02262a5-33b4-4d2d-a0a6-93dad6d51d4b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:36:59.990366 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02262a5-33b4-4d2d-a0a6-93dad6d51d4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:00.091707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02262a5-33b4-4d2d-a0a6-93dad6d51d4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:00.093622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02262a5-33b4-4d2d-a0a6-93dad6d51d4b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:00.238393 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 101/206] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:36:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:00.244003 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4fc37-c5c1-4eb7-9773-d8dcbd8e560b None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:00.331898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:00.366833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4fc37-c5c1-4eb7-9773-d8dcbd8e560b None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:00.413772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dddeb5-14e3-4b42-be5b-7c0d980e1603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:00.414454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dddeb5-14e3-4b42-be5b-7c0d980e1603 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:00.414742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dddeb5-14e3-4b42-be5b-7c0d980e1603 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:00.415025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dddeb5-14e3-4b42-be5b-7c0d980e1603 None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:00.416235 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dddeb5-14e3-4b42-be5b-7c0d980e1603 None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:00.418375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dddeb5-14e3-4b42-be5b-7c0d980e1603 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:00.420401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 106/207] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:00 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:00.429361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02262a5-33b4-4d2d-a0a6-93dad6d51d4b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:00.461287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:00.575475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02262a5-33b4-4d2d-a0a6-93dad6d51d4b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:00.613495 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0ecd6-53bc-4073-84da-e64661007982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:00.614548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0ecd6-53bc-4073-84da-e64661007982 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:00.615022 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0ecd6-53bc-4073-84da-e64661007982 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:00.615485 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0ecd6-53bc-4073-84da-e64661007982 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:00.616498 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0ecd6-53bc-4073-84da-e64661007982 None admin] RBAC: Adding query filter params (role.id=5f5e6a876c4a4fb0869e5bea07e7a65c, scope.project.id=42291678aa1842ff9db31b9d1e39dc2c, user.id=af8505067983430097b469a5330dbcb3) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:00.617018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0ecd6-53bc-4073-84da-e64661007982 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:00.618655 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0ecd6-53bc-4073-84da-e64661007982 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:00.626205 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 102/208] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:00 2022] GET /identity/v3/role_assignments?user.id=af8505067983430097b469a5330dbcb3&scope.project.id=42291678aa1842ff9db31b9d1e39dc2c&role.id=5f5e6a876c4a4fb0869e5bea07e7a65c => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:01.364064 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c623fd1-b15b-4bbe-918e-d87555388bcc None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:01.365126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c623fd1-b15b-4bbe-918e-d87555388bcc None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:01.365633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c623fd1-b15b-4bbe-918e-d87555388bcc None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:01.367412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 107/209] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:01.373842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd26a5-f3a9-4537-9a5e-675b52b57f8c None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:01.374700 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd26a5-f3a9-4537-9a5e-675b52b57f8c None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:01.375187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd26a5-f3a9-4537-9a5e-675b52b57f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:01.394981 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-cacd26a5-f3a9-4537-9a5e-675b52b57f8c None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:01.396536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-cacd26a5-f3a9-4537-9a5e-675b52b57f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:01.419861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f3438-19dd-4642-b8a8-28c3d19a5563 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:01.420170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f3438-19dd-4642-b8a8-28c3d19a5563 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:01.420489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f3438-19dd-4642-b8a8-28c3d19a5563 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:01.421854 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 108/210] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:01.427203 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed4dbc-a6f9-4ed7-91af-7ddc55e99152 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:01.427689 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed4dbc-a6f9-4ed7-91af-7ddc55e99152 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:01.428063 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed4dbc-a6f9-4ed7-91af-7ddc55e99152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:01.447006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-e2ed4dbc-a6f9-4ed7-91af-7ddc55e99152 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:01.449228 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e2ed4dbc-a6f9-4ed7-91af-7ddc55e99152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:01.485242 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 103/211] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:01.508718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd26a5-f3a9-4537-9a5e-675b52b57f8c None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:01.521999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:01.530278 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 109/212] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:01.552658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed4dbc-a6f9-4ed7-91af-7ddc55e99152 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:01.621208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd26a5-f3a9-4537-9a5e-675b52b57f8c None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:01.637641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:01.658020 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165b5ed-cb7e-481e-a461-13ca23d742ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:01.658548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165b5ed-cb7e-481e-a461-13ca23d742ed None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:01.658833 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165b5ed-cb7e-481e-a461-13ca23d742ed None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:01.659110 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165b5ed-cb7e-481e-a461-13ca23d742ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:01.668482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed4dbc-a6f9-4ed7-91af-7ddc55e99152 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:01.670096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165b5ed-cb7e-481e-a461-13ca23d742ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-8165b5ed-cb7e-481e-a461-13ca23d742ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:01.675157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:01.676067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:01.676902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:01.676902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:01.676902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:01.676902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:01.676902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:01.676902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:01.676902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:01.676902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:01.676902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:01.676902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:01.676902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:01.676902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:01.676902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 104/213] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:01.682094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165b5ed-cb7e-481e-a461-13ca23d742ed None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:01.709453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2b47c-e970-420e-a93d-215594cc6b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:01.710442 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2b47c-e970-420e-a93d-215594cc6b47 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:01.710978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2b47c-e970-420e-a93d-215594cc6b47 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:01.711557 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2b47c-e970-420e-a93d-215594cc6b47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:01.725512 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2b47c-e970-420e-a93d-215594cc6b47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-29e2b47c-e970-420e-a93d-215594cc6b47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:01.731730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:01.732663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:01.733454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:01.733454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:01.733454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:01.733454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:01.733454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:01.733454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:01.733454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:01.733454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:01.733454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:01.733454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:01.733454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:37:01.733454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:01.740550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 110/214] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:37:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:01.742251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2b47c-e970-420e-a93d-215594cc6b47 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:01.766240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:01.799596 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165b5ed-cb7e-481e-a461-13ca23d742ed None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:01.835965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316c4ed-d249-4d23-972d-c30f87b52699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:01.836620 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316c4ed-d249-4d23-972d-c30f87b52699 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:01.836910 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316c4ed-d249-4d23-972d-c30f87b52699 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:01.837214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316c4ed-d249-4d23-972d-c30f87b52699 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:01.838127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316c4ed-d249-4d23-972d-c30f87b52699 None admin] RBAC: Adding query filter params (name=member) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:01.838245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316c4ed-d249-4d23-972d-c30f87b52699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:01.839806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316c4ed-d249-4d23-972d-c30f87b52699 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:01.850621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 105/215] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:01.854870 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159d943-2114-4130-ad8c-f762f5c0325c None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:01.855280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159d943-2114-4130-ad8c-f762f5c0325c None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:01.855660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159d943-2114-4130-ad8c-f762f5c0325c None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:01.857091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2b47c-e970-420e-a93d-215594cc6b47 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:01.857209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 106/216] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:01.861070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159d943-2114-4130-ad8c-f762f5c0325c None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:01.897598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed370c-7400-4070-b12b-2fe6d5b2578d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:01.898148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed370c-7400-4070-b12b-2fe6d5b2578d None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:01.898432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed370c-7400-4070-b12b-2fe6d5b2578d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:01.898706 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed370c-7400-4070-b12b-2fe6d5b2578d None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:01.899431 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed370c-7400-4070-b12b-2fe6d5b2578d None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:01.899735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed370c-7400-4070-b12b-2fe6d5b2578d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:01.901723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed370c-7400-4070-b12b-2fe6d5b2578d None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:01.903220 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:01.917205 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 111/217] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:01.918784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b46c3e-968e-4bfc-9ef8-e5ecf9d02685 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:01.919074 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b46c3e-968e-4bfc-9ef8-e5ecf9d02685 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:01.919359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b46c3e-968e-4bfc-9ef8-e5ecf9d02685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:01.939172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-06b46c3e-968e-4bfc-9ef8-e5ecf9d02685 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:01.940404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-06b46c3e-968e-4bfc-9ef8-e5ecf9d02685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:01.988003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159d943-2114-4130-ad8c-f762f5c0325c None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:02.015227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 112/218] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:02.016076 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b46c3e-968e-4bfc-9ef8-e5ecf9d02685 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:02.032922 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:02.040127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a2f63-b3e8-45d1-89fe-e1dc9c57ae6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:02.041630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a2f63-b3e8-45d1-89fe-e1dc9c57ae6f None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:02.042407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a2f63-b3e8-45d1-89fe-e1dc9c57ae6f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:02.043516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a2f63-b3e8-45d1-89fe-e1dc9c57ae6f None admin] PATH_INFO: `/v3/users/76c1816559674b8d849538aa34d3716b` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:02.057929 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a2f63-b3e8-45d1-89fe-e1dc9c57ae6f None admin] RBAC: Authorizing `identity:get_user(user_id=76c1816559674b8d849538aa34d3716b)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:02.059578 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a2f63-b3e8-45d1-89fe-e1dc9c57ae6f None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:02.061446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 107/219] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:01 2022] GET /identity/v3/users/76c1816559674b8d849538aa34d3716b => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:02.062298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a2f63-b3e8-45d1-89fe-e1dc9c57ae6f None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:02.135179 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b46c3e-968e-4bfc-9ef8-e5ecf9d02685 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:02.163359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:02.172441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a2f63-b3e8-45d1-89fe-e1dc9c57ae6f None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:02.174565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b052c-d0e7-4b3b-9721-ef62ffe9ebba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:02.175489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b052c-d0e7-4b3b-9721-ef62ffe9ebba None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:02.175489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b052c-d0e7-4b3b-9721-ef62ffe9ebba None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:02.175773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b052c-d0e7-4b3b-9721-ef62ffe9ebba None admin] PATH_INFO: `/v3/users/a4bf2d9337214f06a8c625d58661fb71` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:02.176985 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b052c-d0e7-4b3b-9721-ef62ffe9ebba None admin] RBAC: Authorizing `identity:get_user(user_id=a4bf2d9337214f06a8c625d58661fb71)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:02.178473 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b052c-d0e7-4b3b-9721-ef62ffe9ebba None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:02.180388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 113/220] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:02 2022] GET /identity/v3/users/a4bf2d9337214f06a8c625d58661fb71 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:02.181877 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b052c-d0e7-4b3b-9721-ef62ffe9ebba None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:02.210194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91b2cc-3d11-477d-8276-4492546f41f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:02.210847 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91b2cc-3d11-477d-8276-4492546f41f9 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:02.211216 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91b2cc-3d11-477d-8276-4492546f41f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:02.211474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91b2cc-3d11-477d-8276-4492546f41f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:02.212643 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91b2cc-3d11-477d-8276-4492546f41f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:02.214434 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91b2cc-3d11-477d-8276-4492546f41f9 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:02.220670 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 108/221] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:02.220670 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91b2cc-3d11-477d-8276-4492546f41f9 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:02.286232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:02.297894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b052c-d0e7-4b3b-9721-ef62ffe9ebba None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:02.332100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91b2cc-3d11-477d-8276-4492546f41f9 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:02.334815 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43921fd7-85c4-46df-8385-6745c1463e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:02.335448 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43921fd7-85c4-46df-8385-6745c1463e00 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:02.335765 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43921fd7-85c4-46df-8385-6745c1463e00 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:02.336046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43921fd7-85c4-46df-8385-6745c1463e00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:02.370069 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e02c66-e3d6-4855-ae16-758e704eea63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:02.370864 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e02c66-e3d6-4855-ae16-758e704eea63 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:02.370993 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e02c66-e3d6-4855-ae16-758e704eea63 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:02.371322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e02c66-e3d6-4855-ae16-758e704eea63 None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:02.372485 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e02c66-e3d6-4855-ae16-758e704eea63 None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:02.373915 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e02c66-e3d6-4855-ae16-758e704eea63 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:02.380555 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 109/222] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:02 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:02.380555 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e02c66-e3d6-4855-ae16-758e704eea63 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:02.404741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:02.413510 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43921fd7-85c4-46df-8385-6745c1463e00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:02.414922 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43921fd7-85c4-46df-8385-6745c1463e00 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:02.489345 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e02c66-e3d6-4855-ae16-758e704eea63 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:02.515521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:02.522153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 114/223] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:02.522989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43921fd7-85c4-46df-8385-6745c1463e00 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:02.529153 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c70e56-dad0-41d9-9ca4-2f660250ecfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:02.529793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c70e56-dad0-41d9-9ca4-2f660250ecfb None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:02.530075 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c70e56-dad0-41d9-9ca4-2f660250ecfb None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:02.530353 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c70e56-dad0-41d9-9ca4-2f660250ecfb None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:02.531050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c70e56-dad0-41d9-9ca4-2f660250ecfb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:02.532619 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c70e56-dad0-41d9-9ca4-2f660250ecfb None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:02.554958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 110/224] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:37:02 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:37:02.557888 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd2ca7-1d49-41fb-93b6-16158e9f43d9 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:02.558660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd2ca7-1d49-41fb-93b6-16158e9f43d9 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:02.559139 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd2ca7-1d49-41fb-93b6-16158e9f43d9 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:02.560494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 111/225] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:02.561273 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd2ca7-1d49-41fb-93b6-16158e9f43d9 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:02.641704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:02.651941 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43921fd7-85c4-46df-8385-6745c1463e00 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:02.676331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd2ca7-1d49-41fb-93b6-16158e9f43d9 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:02.691256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ad290-77ea-4046-9cf0-959a4fc082d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:02.691941 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ad290-77ea-4046-9cf0-959a4fc082d8 None admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:02.692227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ad290-77ea-4046-9cf0-959a4fc082d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:02.692537 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ad290-77ea-4046-9cf0-959a4fc082d8 None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320/users/a4bf2d9337214f06a8c625d58661fb71/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:02.694578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ad290-77ea-4046-9cf0-959a4fc082d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=15984787df4448da8a1beaa8b7b15320, user_id=a4bf2d9337214f06a8c625d58661fb71, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:02.696112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ad290-77ea-4046-9cf0-959a4fc082d8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:02.711742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 115/226] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:37:02 2022] PUT /identity/v3/projects/15984787df4448da8a1beaa8b7b15320/users/a4bf2d9337214f06a8c625d58661fb71/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:37:02.713308 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290e5ae-853a-4837-bcf5-58002a514a66 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:02.714348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290e5ae-853a-4837-bcf5-58002a514a66 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:02.715021 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290e5ae-853a-4837-bcf5-58002a514a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:02.727702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95db49b-41af-478a-b08f-64442b9fbc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:02.728459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95db49b-41af-478a-b08f-64442b9fbc30 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:02.728911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95db49b-41af-478a-b08f-64442b9fbc30 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:02.729774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95db49b-41af-478a-b08f-64442b9fbc30 None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:02.731638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95db49b-41af-478a-b08f-64442b9fbc30 None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:02.733124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95db49b-41af-478a-b08f-64442b9fbc30 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:02.735160 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 112/227] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:02 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:02.736118 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-8290e5ae-853a-4837-bcf5-58002a514a66 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:02.737402 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8290e5ae-853a-4837-bcf5-58002a514a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:02.741465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95db49b-41af-478a-b08f-64442b9fbc30 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:02.817990 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:02.832254 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 116/228] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:02.854979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290e5ae-853a-4837-bcf5-58002a514a66 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:02.886088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95db49b-41af-478a-b08f-64442b9fbc30 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:02.941600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651df10-a440-440d-820d-42ffde55d80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:02.942244 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651df10-a440-440d-820d-42ffde55d80c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:02.942526 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651df10-a440-440d-820d-42ffde55d80c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:02.942808 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651df10-a440-440d-820d-42ffde55d80c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:02.943624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651df10-a440-440d-820d-42ffde55d80c None admin] RBAC: Adding query filter params (role.id=f3b7ffe4d1f04af18f7da8d9a51031b3, scope.project.id=15984787df4448da8a1beaa8b7b15320, user.id=76c1816559674b8d849538aa34d3716b) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:02.943936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651df10-a440-440d-820d-42ffde55d80c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:02.945441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651df10-a440-440d-820d-42ffde55d80c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:02.954125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 113/229] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:02 2022] GET /identity/v3/role_assignments?user.id=76c1816559674b8d849538aa34d3716b&scope.project.id=15984787df4448da8a1beaa8b7b15320&role.id=f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:02.986001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:03.005062 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290e5ae-853a-4837-bcf5-58002a514a66 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:03.045372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0aee4f-97d3-464e-bc5b-664c4b76ec8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:03.046429 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0aee4f-97d3-464e-bc5b-664c4b76ec8b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:03.046918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0aee4f-97d3-464e-bc5b-664c4b76ec8b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:03.047395 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0aee4f-97d3-464e-bc5b-664c4b76ec8b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:03.060867 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0aee4f-97d3-464e-bc5b-664c4b76ec8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-ab0aee4f-97d3-464e-bc5b-664c4b76ec8b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:03.066306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:03.067208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:03.068006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:03.068006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:03.068006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:03.068006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:03.068006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:03.068006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:03.068006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:03.068006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:03.068006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:03.068006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:03.068006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:03.068006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:03.070461 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 117/230] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:37:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:03.079010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651df10-a440-440d-820d-42ffde55d80c None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:03.239995 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651df10-a440-440d-820d-42ffde55d80c None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:03.256109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:03.279627 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669d2b8-1df6-4d6c-bd34-7737c63216cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:03.280268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669d2b8-1df6-4d6c-bd34-7737c63216cc None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:03.280573 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669d2b8-1df6-4d6c-bd34-7737c63216cc None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:03.280849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669d2b8-1df6-4d6c-bd34-7737c63216cc None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:03.281582 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669d2b8-1df6-4d6c-bd34-7737c63216cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:03.281888 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669d2b8-1df6-4d6c-bd34-7737c63216cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:03.283261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669d2b8-1df6-4d6c-bd34-7737c63216cc None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:03.293197 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 114/231] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:03.301392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0aee4f-97d3-464e-bc5b-664c4b76ec8b None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:03.446998 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0aee4f-97d3-464e-bc5b-664c4b76ec8b None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:03.501077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f65a70-5284-4d0a-a92e-c1ad2fe040cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:03.502076 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f65a70-5284-4d0a-a92e-c1ad2fe040cf None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:03.502557 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f65a70-5284-4d0a-a92e-c1ad2fe040cf None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:03.503025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f65a70-5284-4d0a-a92e-c1ad2fe040cf None admin] PATH_INFO: `/v3/users/a4bf2d9337214f06a8c625d58661fb71` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:03.504892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f65a70-5284-4d0a-a92e-c1ad2fe040cf None admin] RBAC: Authorizing `identity:get_user(user_id=a4bf2d9337214f06a8c625d58661fb71)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:03.506592 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f65a70-5284-4d0a-a92e-c1ad2fe040cf None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:03.508409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 118/232] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:03 2022] GET /identity/v3/users/a4bf2d9337214f06a8c625d58661fb71 => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:03.513968 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669d2b8-1df6-4d6c-bd34-7737c63216cc None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:03.661945 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669d2b8-1df6-4d6c-bd34-7737c63216cc None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:03.711784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66cc4dd-79a1-496a-8378-bef42f0cd485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:03.712471 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cc4dd-79a1-496a-8378-bef42f0cd485 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:03.712759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cc4dd-79a1-496a-8378-bef42f0cd485 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:03.713037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cc4dd-79a1-496a-8378-bef42f0cd485 None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:03.714354 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66cc4dd-79a1-496a-8378-bef42f0cd485 None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:03.715884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66cc4dd-79a1-496a-8378-bef42f0cd485 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:03.717941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 115/233] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:03 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:03.723290 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f65a70-5284-4d0a-a92e-c1ad2fe040cf None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:03.749411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:03.869070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f65a70-5284-4d0a-a92e-c1ad2fe040cf None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:03.921152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e6a8a-f58d-4fd8-8427-f152fca01518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:03.921842 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e6a8a-f58d-4fd8-8427-f152fca01518 None admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:03.922129 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e6a8a-f58d-4fd8-8427-f152fca01518 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:03.922407 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e6a8a-f58d-4fd8-8427-f152fca01518 None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320/users/a4bf2d9337214f06a8c625d58661fb71/roles/548e622a9ae745d99d4cd34e68c1652c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:03.926084 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e6a8a-f58d-4fd8-8427-f152fca01518 None admin] RBAC: Authorizing `identity:create_grant(project_id=15984787df4448da8a1beaa8b7b15320, user_id=a4bf2d9337214f06a8c625d58661fb71, role_id=548e622a9ae745d99d4cd34e68c1652c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:03.927762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e6a8a-f58d-4fd8-8427-f152fca01518 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:03.956038 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 119/234] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:37:03 2022] PUT /identity/v3/projects/15984787df4448da8a1beaa8b7b15320/users/a4bf2d9337214f06a8c625d58661fb71/roles/548e622a9ae745d99d4cd34e68c1652c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:37:05.140765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32056c2b-0939-4f05-9eb8-16f3862b7161 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:05.141301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32056c2b-0939-4f05-9eb8-16f3862b7161 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:05.141301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32056c2b-0939-4f05-9eb8-16f3862b7161 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:05.142919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 116/235] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:05.148140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba036c0-a7e7-4fec-9185-bb8efeb005d8 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:05.149009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba036c0-a7e7-4fec-9185-bb8efeb005d8 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:05.149485 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba036c0-a7e7-4fec-9185-bb8efeb005d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:05.169202 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-2ba036c0-a7e7-4fec-9185-bb8efeb005d8 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:05.170748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2ba036c0-a7e7-4fec-9185-bb8efeb005d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:05.265342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 120/236] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:05.310313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32056c2b-0939-4f05-9eb8-16f3862b7161 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:05.381777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:05.449724 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32056c2b-0939-4f05-9eb8-16f3862b7161 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:05.486922 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58950f35-9cb3-4fe7-83e6-4a326b776677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:05.487924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58950f35-9cb3-4fe7-83e6-4a326b776677 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:05.488481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58950f35-9cb3-4fe7-83e6-4a326b776677 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:05.488956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58950f35-9cb3-4fe7-83e6-4a326b776677 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:05.500756 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58950f35-9cb3-4fe7-83e6-4a326b776677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-58950f35-9cb3-4fe7-83e6-4a326b776677 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:05.506257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:05.507131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:05.507947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:05.507947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:05.507947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:05.507947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:05.507947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:05.507947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:05.507947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:05.507947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:05.507947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:05.507947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:05.507947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:37:05.507947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:05.510976 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 117/237] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:37:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:05.515843 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d7bbd-555b-4c2e-80cf-0bb002420e5b None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:05.516417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d7bbd-555b-4c2e-80cf-0bb002420e5b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:05.516895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d7bbd-555b-4c2e-80cf-0bb002420e5b None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:05.517235 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58950f35-9cb3-4fe7-83e6-4a326b776677 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:05.519460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 121/238] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:05.526753 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4419867-2e04-4c0b-b85f-8afc95664cf3 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:05.527315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4419867-2e04-4c0b-b85f-8afc95664cf3 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:05.527802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4419867-2e04-4c0b-b85f-8afc95664cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:05.542658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:05.559753 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-b4419867-2e04-4c0b-b85f-8afc95664cf3 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:05.562156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b4419867-2e04-4c0b-b85f-8afc95664cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:05.658751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 122/239] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:05.660618 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df3bc1-b7cc-424a-b9ff-34c0763d20ac None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:05.661107 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df3bc1-b7cc-424a-b9ff-34c0763d20ac None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:05.661575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df3bc1-b7cc-424a-b9ff-34c0763d20ac None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:05.662548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58950f35-9cb3-4fe7-83e6-4a326b776677 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:05.667639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 123/240] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:05.669189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbc211-a7e7-4be0-8d3d-b9d5bec16cca None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:05.669748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbc211-a7e7-4be0-8d3d-b9d5bec16cca None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:05.670221 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbc211-a7e7-4be0-8d3d-b9d5bec16cca None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:05.671705 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 124/241] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:05.673425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6ebe-1a07-4cf1-9a5f-fae47d242053 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:05.673908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6ebe-1a07-4cf1-9a5f-fae47d242053 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:05.674398 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6ebe-1a07-4cf1-9a5f-fae47d242053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:05.680783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:05.695564 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-e58d6ebe-1a07-4cf1-9a5f-fae47d242053 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:05.696792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e58d6ebe-1a07-4cf1-9a5f-fae47d242053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:05.700667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393452f0-1177-49b4-9e6b-4e442595be89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:05.701297 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393452f0-1177-49b4-9e6b-4e442595be89 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:05.701586 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393452f0-1177-49b4-9e6b-4e442595be89 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:05.701860 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393452f0-1177-49b4-9e6b-4e442595be89 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:05.702585 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393452f0-1177-49b4-9e6b-4e442595be89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:05.702891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393452f0-1177-49b4-9e6b-4e442595be89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:05.704264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393452f0-1177-49b4-9e6b-4e442595be89 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:05.718468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 118/242] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:05.718468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70012e83-1256-4b9a-97ff-e1e4f3928ee2 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:05.718468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70012e83-1256-4b9a-97ff-e1e4f3928ee2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:05.718468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70012e83-1256-4b9a-97ff-e1e4f3928ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:05.735214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-70012e83-1256-4b9a-97ff-e1e4f3928ee2 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:05.736428 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-70012e83-1256-4b9a-97ff-e1e4f3928ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:05.773351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 125/243] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:05.785491 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d6ebe-1a07-4cf1-9a5f-fae47d242053 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:05.812217 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 119/244] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:05.817965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70012e83-1256-4b9a-97ff-e1e4f3928ee2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:05.852996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:05.911657 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d6ebe-1a07-4cf1-9a5f-fae47d242053 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:05.934579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:05.944710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70012e83-1256-4b9a-97ff-e1e4f3928ee2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:05.957810 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074192d-ba01-4cab-b166-70c520518cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:05.958456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074192d-ba01-4cab-b166-70c520518cac None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:05.958735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074192d-ba01-4cab-b166-70c520518cac None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:05.959010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074192d-ba01-4cab-b166-70c520518cac None admin] PATH_INFO: `/v3/roles/member` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:05.971369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074192d-ba01-4cab-b166-70c520518cac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-0074192d-ba01-4cab-b166-70c520518cac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:05.976263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:05.977700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:05.978913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:05.978913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:05.978913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:05.978913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:05.978913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:05.978913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:05.978913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:05.978913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:05.978913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:05.978913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:05.978913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:05.978913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:05.978913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 126/245] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:05.978913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074192d-ba01-4cab-b166-70c520518cac None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:06.001003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee29d5-079c-4280-9dff-d0b092bf433b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:06.001654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee29d5-079c-4280-9dff-d0b092bf433b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:06.001934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee29d5-079c-4280-9dff-d0b092bf433b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:06.002206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee29d5-079c-4280-9dff-d0b092bf433b None admin] PATH_INFO: `/v3/users/a4bf2d9337214f06a8c625d58661fb71` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:06.003391 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee29d5-079c-4280-9dff-d0b092bf433b None admin] RBAC: Authorizing `identity:get_user(user_id=a4bf2d9337214f06a8c625d58661fb71)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:06.005109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee29d5-079c-4280-9dff-d0b092bf433b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:06.006709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 120/246] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:05 2022] GET /identity/v3/users/a4bf2d9337214f06a8c625d58661fb71 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:06.007459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee29d5-079c-4280-9dff-d0b092bf433b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:06.109196 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074192d-ba01-4cab-b166-70c520518cac None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:06.118772 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee29d5-079c-4280-9dff-d0b092bf433b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:06.139380 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:06.152526 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c10fa9-8df7-4e72-bde1-4293e65aec13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:06.153133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c10fa9-8df7-4e72-bde1-4293e65aec13 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:06.153431 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c10fa9-8df7-4e72-bde1-4293e65aec13 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:06.153707 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c10fa9-8df7-4e72-bde1-4293e65aec13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:06.154858 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c10fa9-8df7-4e72-bde1-4293e65aec13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:06.156263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbd517-709d-471a-ac63-f65def55261b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:06.156654 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c10fa9-8df7-4e72-bde1-4293e65aec13 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:06.156941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbd517-709d-471a-ac63-f65def55261b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:06.157226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbd517-709d-471a-ac63-f65def55261b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:06.157586 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbd517-709d-471a-ac63-f65def55261b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:06.162655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 127/247] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:06.163431 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c10fa9-8df7-4e72-bde1-4293e65aec13 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:06.170594 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbd517-709d-471a-ac63-f65def55261b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-3acbd517-709d-471a-ac63-f65def55261b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:06.176854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:06.177742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:06.178537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:06.178537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:06.178537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:06.178537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:06.178537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:06.178537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:06.178537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:06.178537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:06.178537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:06.178537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:06.178537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:37:06.178537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:06.180426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 121/248] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:06.181354 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbd517-709d-471a-ac63-f65def55261b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:06.211458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:06.282057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c10fa9-8df7-4e72-bde1-4293e65aec13 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:06.295159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbd517-709d-471a-ac63-f65def55261b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:06.322404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dff05-9928-4607-a438-85aebee7e13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:06.323244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dff05-9928-4607-a438-85aebee7e13b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:06.323527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dff05-9928-4607-a438-85aebee7e13b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:06.323805 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dff05-9928-4607-a438-85aebee7e13b None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:06.324571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5dff05-9928-4607-a438-85aebee7e13b None admin] RBAC: Adding query filter params (name=member) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:06.325109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5dff05-9928-4607-a438-85aebee7e13b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:06.326263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5dff05-9928-4607-a438-85aebee7e13b None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:06.334810 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 128/249] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:06.335400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dff05-9928-4607-a438-85aebee7e13b None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:06.340531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798a566-6239-42d2-9eb2-9b062e7b3235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:06.341179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798a566-6239-42d2-9eb2-9b062e7b3235 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:06.341458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798a566-6239-42d2-9eb2-9b062e7b3235 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:06.341721 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798a566-6239-42d2-9eb2-9b062e7b3235 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:06.391833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:06.413128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798a566-6239-42d2-9eb2-9b062e7b3235 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:06.414598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798a566-6239-42d2-9eb2-9b062e7b3235 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:06.438041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:06.438688 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dff05-9928-4607-a438-85aebee7e13b None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:06.474401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d2196-ea54-4593-9907-5947ed946038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:06.475579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d2196-ea54-4593-9907-5947ed946038 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:06.475870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d2196-ea54-4593-9907-5947ed946038 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:06.476144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d2196-ea54-4593-9907-5947ed946038 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:06.476863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d2196-ea54-4593-9907-5947ed946038 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:06.478322 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d2196-ea54-4593-9907-5947ed946038 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:06.500120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 129/250] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:37:06 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:37:06.500994 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d2196-ea54-4593-9907-5947ed946038 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:06.523606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 122/251] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:06.524649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798a566-6239-42d2-9eb2-9b062e7b3235 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:06.618112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d2196-ea54-4593-9907-5947ed946038 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:06.632424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:06.654536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798a566-6239-42d2-9eb2-9b062e7b3235 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:06.663215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51903202-6bc2-4638-bec1-05dfceb71313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:06.663851 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51903202-6bc2-4638-bec1-05dfceb71313 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:06.664139 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51903202-6bc2-4638-bec1-05dfceb71313 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:06.664445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51903202-6bc2-4638-bec1-05dfceb71313 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:06.665182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51903202-6bc2-4638-bec1-05dfceb71313 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:06.665489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51903202-6bc2-4638-bec1-05dfceb71313 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:06.667247 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51903202-6bc2-4638-bec1-05dfceb71313 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:06.677854 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 130/252] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:06.678964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f526c82-c3e6-41db-8b30-209e08a11c06 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:06.679256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f526c82-c3e6-41db-8b30-209e08a11c06 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:06.679533 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f526c82-c3e6-41db-8b30-209e08a11c06 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:06.680802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 131/253] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:06.681504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f526c82-c3e6-41db-8b30-209e08a11c06 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:06.687340 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:06.701006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3f71f-f110-4b36-ac38-5c12bfc43f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:06.701654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3f71f-f110-4b36-ac38-5c12bfc43f49 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:06.701940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3f71f-f110-4b36-ac38-5c12bfc43f49 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:06.702212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3f71f-f110-4b36-ac38-5c12bfc43f49 None admin] PATH_INFO: `/v3/users/76c1816559674b8d849538aa34d3716b` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:06.703651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3f71f-f110-4b36-ac38-5c12bfc43f49 None admin] RBAC: Authorizing `identity:get_user(user_id=76c1816559674b8d849538aa34d3716b)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:06.705209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3f71f-f110-4b36-ac38-5c12bfc43f49 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:06.707250 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 123/254] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:06 2022] GET /identity/v3/users/76c1816559674b8d849538aa34d3716b => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:06.708085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3f71f-f110-4b36-ac38-5c12bfc43f49 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:06.817394 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f526c82-c3e6-41db-8b30-209e08a11c06 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:06.841308 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3f71f-f110-4b36-ac38-5c12bfc43f49 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:06.871826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7dd9e-fd23-497f-9914-e6beaaeb7656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:06.873083 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7dd9e-fd23-497f-9914-e6beaaeb7656 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:06.873818 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7dd9e-fd23-497f-9914-e6beaaeb7656 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:06.874277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7dd9e-fd23-497f-9914-e6beaaeb7656 None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:06.876477 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7dd9e-fd23-497f-9914-e6beaaeb7656 None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:06.879027 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7dd9e-fd23-497f-9914-e6beaaeb7656 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:06.881897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 132/255] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:06 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:06.883334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44861884-f28d-49cc-b644-4b34dce4d0cb None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:06.883465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59812265-3b51-4b36-8b4a-612c7f3a0b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:06.883935 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44861884-f28d-49cc-b644-4b34dce4d0cb None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:06.884411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44861884-f28d-49cc-b644-4b34dce4d0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:06.884683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59812265-3b51-4b36-8b4a-612c7f3a0b06 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:06.885322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59812265-3b51-4b36-8b4a-612c7f3a0b06 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:06.886022 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59812265-3b51-4b36-8b4a-612c7f3a0b06 None admin] PATH_INFO: `/v3/users/b185426ee58b4d9ca878fab3369ddef7` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:06.902379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59812265-3b51-4b36-8b4a-612c7f3a0b06 None admin] RBAC: Authorizing `identity:get_user(user_id=b185426ee58b4d9ca878fab3369ddef7)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:06.903758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59812265-3b51-4b36-8b4a-612c7f3a0b06 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:06.905953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 124/256] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:06 2022] GET /identity/v3/users/b185426ee58b4d9ca878fab3369ddef7 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:06.906739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59812265-3b51-4b36-8b4a-612c7f3a0b06 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:06.908530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-44861884-f28d-49cc-b644-4b34dce4d0cb None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:06.910261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-44861884-f28d-49cc-b644-4b34dce4d0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:06.954338 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:06.965640 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:06.998648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 133/257] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:06.999672 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44861884-f28d-49cc-b644-4b34dce4d0cb None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:07.035862 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59812265-3b51-4b36-8b4a-612c7f3a0b06 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:07.083885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8511f-ed38-45b4-8253-44dfb6d48bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:07.084795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8511f-ed38-45b4-8253-44dfb6d48bfc None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:07.085269 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8511f-ed38-45b4-8253-44dfb6d48bfc None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:07.085741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8511f-ed38-45b4-8253-44dfb6d48bfc None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:07.087840 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8511f-ed38-45b4-8253-44dfb6d48bfc None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:07.089659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8511f-ed38-45b4-8253-44dfb6d48bfc None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:07.091378 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 125/258] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:06 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:07.096835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8511f-ed38-45b4-8253-44dfb6d48bfc None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:07.117613 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44861884-f28d-49cc-b644-4b34dce4d0cb None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:07.173117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719b4de-c3da-4f10-9d0f-e2dfc24b9fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:07.173721 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719b4de-c3da-4f10-9d0f-e2dfc24b9fb8 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:07.174017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719b4de-c3da-4f10-9d0f-e2dfc24b9fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:07.174352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719b4de-c3da-4f10-9d0f-e2dfc24b9fb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:07.175491 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719b4de-c3da-4f10-9d0f-e2dfc24b9fb8 None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=15984787df4448da8a1beaa8b7b15320, user.id=a4bf2d9337214f06a8c625d58661fb71) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:07.175799 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719b4de-c3da-4f10-9d0f-e2dfc24b9fb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:07.177879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719b4de-c3da-4f10-9d0f-e2dfc24b9fb8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:07.185864 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 134/259] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:06 2022] GET /identity/v3/role_assignments?user.id=a4bf2d9337214f06a8c625d58661fb71&scope.project.id=15984787df4448da8a1beaa8b7b15320&role.id=19557e941d6f475ba92cf862846f9872 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:07.186676 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719b4de-c3da-4f10-9d0f-e2dfc24b9fb8 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:07.223133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8511f-ed38-45b4-8253-44dfb6d48bfc None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:07.232767 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:07.252042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:07.278923 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23402816-4619-49fd-8942-12dc077f6e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:07.280227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23402816-4619-49fd-8942-12dc077f6e08 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:07.281481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23402816-4619-49fd-8942-12dc077f6e08 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:07.282012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23402816-4619-49fd-8942-12dc077f6e08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:07.320208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719b4de-c3da-4f10-9d0f-e2dfc24b9fb8 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:07.358595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a0400-02a1-4796-bb13-faaed2a4c120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:07.359296 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a0400-02a1-4796-bb13-faaed2a4c120 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:07.359577 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a0400-02a1-4796-bb13-faaed2a4c120 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:07.359850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a0400-02a1-4796-bb13-faaed2a4c120 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:07.372631 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a0400-02a1-4796-bb13-faaed2a4c120 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-948a0400-02a1-4796-bb13-faaed2a4c120 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:07.378288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:07.379481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:07.380307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:07.380307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:07.380307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:07.380307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:07.380307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:07.380307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:07.380307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:07.380307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:07.380307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:07.380307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:07.380307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:07.380307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:07.380307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 135/260] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:37:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:07.384545 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a0400-02a1-4796-bb13-faaed2a4c120 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:07.386096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23402816-4619-49fd-8942-12dc077f6e08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:07.387814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23402816-4619-49fd-8942-12dc077f6e08 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:07.508574 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:07.523375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a0400-02a1-4796-bb13-faaed2a4c120 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:07.564031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 126/261] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:07.572308 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23402816-4619-49fd-8942-12dc077f6e08 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:07.577079 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc608db-6c05-4787-afd2-272622e99265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:07.577953 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc608db-6c05-4787-afd2-272622e99265 None admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:07.578412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc608db-6c05-4787-afd2-272622e99265 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:07.578838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc608db-6c05-4787-afd2-272622e99265 None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320/users/76c1816559674b8d849538aa34d3716b/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:07.583860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc608db-6c05-4787-afd2-272622e99265 None admin] RBAC: Authorizing `identity:create_grant(project_id=15984787df4448da8a1beaa8b7b15320, user_id=76c1816559674b8d849538aa34d3716b, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:07.585365 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc608db-6c05-4787-afd2-272622e99265 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:07.597548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:07.611408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 136/262] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:37:07 2022] PUT /identity/v3/projects/15984787df4448da8a1beaa8b7b15320/users/76c1816559674b8d849538aa34d3716b/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:37:07.612268 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc608db-6c05-4787-afd2-272622e99265 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:07.694940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23402816-4619-49fd-8942-12dc077f6e08 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:07.735299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442106e-13b4-4eb2-9f6e-779119edd00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:07.736374 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442106e-13b4-4eb2-9f6e-779119edd00a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:07.736914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442106e-13b4-4eb2-9f6e-779119edd00a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:07.737481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442106e-13b4-4eb2-9f6e-779119edd00a None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:07.738414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442106e-13b4-4eb2-9f6e-779119edd00a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:07.738915 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442106e-13b4-4eb2-9f6e-779119edd00a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:07.740545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442106e-13b4-4eb2-9f6e-779119edd00a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:07.751577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 127/263] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:07.758220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442106e-13b4-4eb2-9f6e-779119edd00a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:07.811876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc608db-6c05-4787-afd2-272622e99265 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:07.854644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0401cea-a38d-46ee-aade-b01920682b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:07.855306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0401cea-a38d-46ee-aade-b01920682b07 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:07.855645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0401cea-a38d-46ee-aade-b01920682b07 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:07.855929 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0401cea-a38d-46ee-aade-b01920682b07 None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:07.857278 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0401cea-a38d-46ee-aade-b01920682b07 None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:07.858745 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0401cea-a38d-46ee-aade-b01920682b07 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:07.859911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:07.861277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 137/264] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:07 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:07.867378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0401cea-a38d-46ee-aade-b01920682b07 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:07.889890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442106e-13b4-4eb2-9f6e-779119edd00a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:07.921297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:07.977893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82bcd9-7ef2-4a74-8afb-5c8cdd2e5a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:07.979007 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82bcd9-7ef2-4a74-8afb-5c8cdd2e5a65 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:07.979497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82bcd9-7ef2-4a74-8afb-5c8cdd2e5a65 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:07.979975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82bcd9-7ef2-4a74-8afb-5c8cdd2e5a65 None admin] PATH_INFO: `/v3/users/a4bf2d9337214f06a8c625d58661fb71` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:07.981422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82bcd9-7ef2-4a74-8afb-5c8cdd2e5a65 None admin] RBAC: Authorizing `identity:get_user(user_id=a4bf2d9337214f06a8c625d58661fb71)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:07.983150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82bcd9-7ef2-4a74-8afb-5c8cdd2e5a65 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:07.993550 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 128/265] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:07 2022] GET /identity/v3/users/a4bf2d9337214f06a8c625d58661fb71 => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:08.004407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82bcd9-7ef2-4a74-8afb-5c8cdd2e5a65 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:08.089854 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0401cea-a38d-46ee-aade-b01920682b07 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:08.195662 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0239a4-976f-4140-be30-f5c354e988e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:08.195662 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0239a4-976f-4140-be30-f5c354e988e0 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:08.195662 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0239a4-976f-4140-be30-f5c354e988e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:08.197026 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0239a4-976f-4140-be30-f5c354e988e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:08.198830 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0239a4-976f-4140-be30-f5c354e988e0 None admin] RBAC: Adding query filter params (role.id=5f5e6a876c4a4fb0869e5bea07e7a65c, scope.project.id=15984787df4448da8a1beaa8b7b15320, user.id=b185426ee58b4d9ca878fab3369ddef7) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:08.199358 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0239a4-976f-4140-be30-f5c354e988e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:08.203622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0239a4-976f-4140-be30-f5c354e988e0 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:08.222442 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 138/266] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:07 2022] GET /identity/v3/role_assignments?user.id=b185426ee58b4d9ca878fab3369ddef7&scope.project.id=15984787df4448da8a1beaa8b7b15320&role.id=5f5e6a876c4a4fb0869e5bea07e7a65c => generated 259 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:08.275322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82bcd9-7ef2-4a74-8afb-5c8cdd2e5a65 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:08.349411 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa97e7-e127-4a71-aef9-2c23e50be894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:08.350344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa97e7-e127-4a71-aef9-2c23e50be894 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:08.350823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa97e7-e127-4a71-aef9-2c23e50be894 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:08.351293 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa97e7-e127-4a71-aef9-2c23e50be894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:08.447256 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:08.583239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa97e7-e127-4a71-aef9-2c23e50be894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:08.586969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa97e7-e127-4a71-aef9-2c23e50be894 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:08.863193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 129/267] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 859 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 18:37:08.870296 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0239a4-976f-4140-be30-f5c354e988e0 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:09.046738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:09.066167 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0239a4-976f-4140-be30-f5c354e988e0 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:09.070159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b86a58-a471-4b5d-a87f-648de5e919d4 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:09.070453 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b86a58-a471-4b5d-a87f-648de5e919d4 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:09.070732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b86a58-a471-4b5d-a87f-648de5e919d4 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:09.072322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 130/268] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:09.078943 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9055b77-43ac-48d2-b9ce-390da831151b None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:09.079543 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9055b77-43ac-48d2-b9ce-390da831151b None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:09.080021 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9055b77-43ac-48d2-b9ce-390da831151b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:09.116660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f9055b77-43ac-48d2-b9ce-390da831151b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:09.117771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f9055b77-43ac-48d2-b9ce-390da831151b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:09.141443 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325fd18-12e1-4702-95ee-5a4e2e2789b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:09.142095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325fd18-12e1-4702-95ee-5a4e2e2789b6 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:09.142376 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325fd18-12e1-4702-95ee-5a4e2e2789b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:09.142647 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325fd18-12e1-4702-95ee-5a4e2e2789b6 None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:09.144884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325fd18-12e1-4702-95ee-5a4e2e2789b6 None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:09.146961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325fd18-12e1-4702-95ee-5a4e2e2789b6 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:09.148686 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 139/269] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:08 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:09.156481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325fd18-12e1-4702-95ee-5a4e2e2789b6 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:09.225268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:09.287000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 131/270] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:09.345583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325fd18-12e1-4702-95ee-5a4e2e2789b6 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:09.348318 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9055b77-43ac-48d2-b9ce-390da831151b None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:09.427089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306ad0a-bc48-4fd9-9c13-a000cbd1bd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:09.427647 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306ad0a-bc48-4fd9-9c13-a000cbd1bd11 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:09.428033 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306ad0a-bc48-4fd9-9c13-a000cbd1bd11 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:09.428336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306ad0a-bc48-4fd9-9c13-a000cbd1bd11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:09.429161 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306ad0a-bc48-4fd9-9c13-a000cbd1bd11 None admin] RBAC: Adding query filter params (role.id=548e622a9ae745d99d4cd34e68c1652c, scope.project.id=15984787df4448da8a1beaa8b7b15320, user.id=a4bf2d9337214f06a8c625d58661fb71) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:09.429478 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306ad0a-bc48-4fd9-9c13-a000cbd1bd11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:09.431084 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306ad0a-bc48-4fd9-9c13-a000cbd1bd11 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:09.444709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 140/271] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:09 2022] GET /identity/v3/role_assignments?user.id=a4bf2d9337214f06a8c625d58661fb71&scope.project.id=15984787df4448da8a1beaa8b7b15320&role.id=548e622a9ae745d99d4cd34e68c1652c => generated 608 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:09.446239 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dabc9-86d7-4a2f-8b18-2572e4d340f9 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:09.446535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dabc9-86d7-4a2f-8b18-2572e4d340f9 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:09.446817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dabc9-86d7-4a2f-8b18-2572e4d340f9 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:09.456702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 141/272] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:09 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:09.463194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1779ad-d4c5-435e-be58-09adeca50431 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:09.464085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1779ad-d4c5-435e-be58-09adeca50431 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:09.464396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1779ad-d4c5-435e-be58-09adeca50431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:09.501902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9055b77-43ac-48d2-b9ce-390da831151b None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:09.504656 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-7a1779ad-d4c5-435e-be58-09adeca50431 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:09.505884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7a1779ad-d4c5-435e-be58-09adeca50431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:09.529377 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:09.553760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00c37a-601c-4d0e-bd60-fd630d280b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:09.554447 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00c37a-601c-4d0e-bd60-fd630d280b68 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:09.554729 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00c37a-601c-4d0e-bd60-fd630d280b68 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:09.556189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00c37a-601c-4d0e-bd60-fd630d280b68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:09.559251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00c37a-601c-4d0e-bd60-fd630d280b68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:09.560762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00c37a-601c-4d0e-bd60-fd630d280b68 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:09.566653 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 132/273] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:09.572313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00c37a-601c-4d0e-bd60-fd630d280b68 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:09.655115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:09.672377 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 142/274] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 03 18:37:09.695969 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1779ad-d4c5-435e-be58-09adeca50431 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:09.713850 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00c37a-601c-4d0e-bd60-fd630d280b68 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:09.765687 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1f7bd-a16d-4e18-981a-d89b4d9efaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:09.766421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1f7bd-a16d-4e18-981a-d89b4d9efaa1 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:09.767236 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1f7bd-a16d-4e18-981a-d89b4d9efaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:09.767517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1f7bd-a16d-4e18-981a-d89b4d9efaa1 None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:09.768401 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1f7bd-a16d-4e18-981a-d89b4d9efaa1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:09.769999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1f7bd-a16d-4e18-981a-d89b4d9efaa1 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:09.815458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 133/275] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:37:09 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:37:09.837830 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1779ad-d4c5-435e-be58-09adeca50431 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:09.898134 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eb297-473e-499a-b54c-5bb5334e9d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:09.899175 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eb297-473e-499a-b54c-5bb5334e9d01 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:09.899665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eb297-473e-499a-b54c-5bb5334e9d01 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:09.900173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eb297-473e-499a-b54c-5bb5334e9d01 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:09.918156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8eb297-473e-499a-b54c-5bb5334e9d01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-6d8eb297-473e-499a-b54c-5bb5334e9d01 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:09.923371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:09.924343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:09.925597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:09.925597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:09.925597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:09.925597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:09.925597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:09.925597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:09.925597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:09.925597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:09.925597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:09.925597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:09.925597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:09.925597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:09.925597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 143/276] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:09.932020 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1f7bd-a16d-4e18-981a-d89b4d9efaa1 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:10.065775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1f7bd-a16d-4e18-981a-d89b4d9efaa1 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:10.067443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:10.106435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2102bf8-2c39-434d-8ee7-f92b64f50ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:10.107069 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2102bf8-2c39-434d-8ee7-f92b64f50ab4 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:10.107349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2102bf8-2c39-434d-8ee7-f92b64f50ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:10.107626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2102bf8-2c39-434d-8ee7-f92b64f50ab4 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:10.108424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2102bf8-2c39-434d-8ee7-f92b64f50ab4 None admin] RBAC: Adding query filter params (name=member) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:10.108688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2102bf8-2c39-434d-8ee7-f92b64f50ab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:10.110245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2102bf8-2c39-434d-8ee7-f92b64f50ab4 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:10.118977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 134/277] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:10.125950 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eb297-473e-499a-b54c-5bb5334e9d01 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:10.140783 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:10.259435 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eb297-473e-499a-b54c-5bb5334e9d01 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:10.298149 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75098df-7859-4c25-953d-3f156d4112dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:10.298844 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75098df-7859-4c25-953d-3f156d4112dc None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:10.299098 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75098df-7859-4c25-953d-3f156d4112dc None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:10.299378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75098df-7859-4c25-953d-3f156d4112dc None admin] PATH_INFO: `/v3/users/a4740c15087f4f45895b50da687cb884` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:10.315196 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75098df-7859-4c25-953d-3f156d4112dc None admin] RBAC: Authorizing `identity:get_user(user_id=a4740c15087f4f45895b50da687cb884)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:10.316820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75098df-7859-4c25-953d-3f156d4112dc None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:10.318871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 144/278] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:10 2022] GET /identity/v3/users/a4740c15087f4f45895b50da687cb884 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:10.325445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2102bf8-2c39-434d-8ee7-f92b64f50ab4 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:10.423129 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce67ee-ed60-4e72-a1b4-21f9aa1b6796 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:10.424071 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce67ee-ed60-4e72-a1b4-21f9aa1b6796 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:10.424587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce67ee-ed60-4e72-a1b4-21f9aa1b6796 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:10.426400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 145/279] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:10.431536 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee987b7-2318-4283-9eb7-0d392e6052d0 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:10.432852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee987b7-2318-4283-9eb7-0d392e6052d0 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:10.433627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee987b7-2318-4283-9eb7-0d392e6052d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:10.452898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-cee987b7-2318-4283-9eb7-0d392e6052d0 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:10.454580 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-cee987b7-2318-4283-9eb7-0d392e6052d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:10.494884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2102bf8-2c39-434d-8ee7-f92b64f50ab4 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:10.560252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e33b82-96fe-4e22-9dc1-9e9d0ec9a764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:10.561241 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e33b82-96fe-4e22-9dc1-9e9d0ec9a764 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:10.561715 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e33b82-96fe-4e22-9dc1-9e9d0ec9a764 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:10.562204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e33b82-96fe-4e22-9dc1-9e9d0ec9a764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:10.563187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e33b82-96fe-4e22-9dc1-9e9d0ec9a764 None admin] RBAC: Adding query filter params (role.id=f3b7ffe4d1f04af18f7da8d9a51031b3, user.id=a4740c15087f4f45895b50da687cb884, scope.system=all) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:10.563703 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e33b82-96fe-4e22-9dc1-9e9d0ec9a764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:10.566593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e33b82-96fe-4e22-9dc1-9e9d0ec9a764 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:10.573602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:10.576411 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 135/280] 10.222.0.69 () {56 vars in 1308 bytes} [Wed Aug 3 18:37:10 2022] GET /identity/v3/role_assignments?user.id=a4740c15087f4f45895b50da687cb884&scope.system=all&role.id=f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:10.584812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 146/281] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:10.610796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e33b82-96fe-4e22-9dc1-9e9d0ec9a764 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:10.661885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:10.764110 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e33b82-96fe-4e22-9dc1-9e9d0ec9a764 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:10.803694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f21a3-3154-4c9b-8a0c-6fbaf33b7f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:10.804774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f21a3-3154-4c9b-8a0c-6fbaf33b7f5c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:10.805262 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f21a3-3154-4c9b-8a0c-6fbaf33b7f5c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:10.805735 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f21a3-3154-4c9b-8a0c-6fbaf33b7f5c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:10.817154 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f21a3-3154-4c9b-8a0c-6fbaf33b7f5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-021f21a3-3154-4c9b-8a0c-6fbaf33b7f5c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:10.822984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:10.823957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:10.824893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:10.824893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:10.824893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:10.824893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:10.824893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:10.824893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:10.824893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:10.824893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:10.824893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:10.824893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:10.824893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:10.824893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:10.827721 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 136/282] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:10.833496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee987b7-2318-4283-9eb7-0d392e6052d0 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:10.938052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cb357-3f3a-4f63-85b0-d18dd41988b7 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:10.938576 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cb357-3f3a-4f63-85b0-d18dd41988b7 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:10.939079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cb357-3f3a-4f63-85b0-d18dd41988b7 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:10.940959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 137/283] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:10.957797 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d510c39-424b-4287-a829-368c43b420d3 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:10.958404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d510c39-424b-4287-a829-368c43b420d3 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:10.958881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d510c39-424b-4287-a829-368c43b420d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:10.987379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee987b7-2318-4283-9eb7-0d392e6052d0 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:10.988701 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-8d510c39-424b-4287-a829-368c43b420d3 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:10.990102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8d510c39-424b-4287-a829-368c43b420d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:11.027500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538878a-24e9-47ad-a56f-9394c099fd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:11.028095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538878a-24e9-47ad-a56f-9394c099fd7f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:11.028394 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538878a-24e9-47ad-a56f-9394c099fd7f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:11.028670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538878a-24e9-47ad-a56f-9394c099fd7f None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:11.029412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538878a-24e9-47ad-a56f-9394c099fd7f None admin] RBAC: Adding query filter params (name=member) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:11.029717 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538878a-24e9-47ad-a56f-9394c099fd7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:11.031133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538878a-24e9-47ad-a56f-9394c099fd7f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:11.048891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 147/284] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:11.054543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538878a-24e9-47ad-a56f-9394c099fd7f None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:11.116439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 138/285] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:11.148678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:11.151939 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d510c39-424b-4287-a829-368c43b420d3 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:11.182375 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:11.241428 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538878a-24e9-47ad-a56f-9394c099fd7f None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:11.295810 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d510c39-424b-4287-a829-368c43b420d3 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:11.312670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbbb0c-3ebe-4504-b27c-66a575589f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:11.313444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbbb0c-3ebe-4504-b27c-66a575589f3a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:11.313733 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbbb0c-3ebe-4504-b27c-66a575589f3a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:11.314009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbbb0c-3ebe-4504-b27c-66a575589f3a None admin] PATH_INFO: `/v3/users/76c1816559674b8d849538aa34d3716b` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:11.316827 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbbb0c-3ebe-4504-b27c-66a575589f3a None admin] RBAC: Authorizing `identity:get_user(user_id=76c1816559674b8d849538aa34d3716b)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:11.318412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbbb0c-3ebe-4504-b27c-66a575589f3a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:11.320412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 148/286] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:11 2022] GET /identity/v3/users/76c1816559674b8d849538aa34d3716b => generated 297 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:11.338544 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbbb0c-3ebe-4504-b27c-66a575589f3a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:11.344006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c27cd-b0e3-4aa3-b894-e04ef64f7c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:11.344581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c27cd-b0e3-4aa3-b894-e04ef64f7c94 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:11.344815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c27cd-b0e3-4aa3-b894-e04ef64f7c94 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:11.344977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c27cd-b0e3-4aa3-b894-e04ef64f7c94 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:11.368554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c27cd-b0e3-4aa3-b894-e04ef64f7c94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-524c27cd-b0e3-4aa3-b894-e04ef64f7c94 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:11.382661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:11.383604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:11.388959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:11.388959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:11.388959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:11.388959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:11.388959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:11.388959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:11.388959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:11.388959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:11.388959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:11.388959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:11.388959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:37:11.388959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:11.388959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 139/287] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:11.404193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c27cd-b0e3-4aa3-b894-e04ef64f7c94 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:11.467124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbbb0c-3ebe-4504-b27c-66a575589f3a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:11.501314 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:11.514006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b14a2-121d-4f74-aa22-b2a3c254eec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:11.514970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b14a2-121d-4f74-aa22-b2a3c254eec4 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:11.515449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b14a2-121d-4f74-aa22-b2a3c254eec4 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:11.515916 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b14a2-121d-4f74-aa22-b2a3c254eec4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:11.543622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:11.589478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c27cd-b0e3-4aa3-b894-e04ef64f7c94 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:11.633173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b14a2-121d-4f74-aa22-b2a3c254eec4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:11.636354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b14a2-121d-4f74-aa22-b2a3c254eec4 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:11.655173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f577a97-aadd-49c6-87e3-b0f9d5b72549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:11.656232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f577a97-aadd-49c6-87e3-b0f9d5b72549 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:11.656977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f577a97-aadd-49c6-87e3-b0f9d5b72549 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:11.657482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f577a97-aadd-49c6-87e3-b0f9d5b72549 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:11.658421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f577a97-aadd-49c6-87e3-b0f9d5b72549 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:11.661212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f577a97-aadd-49c6-87e3-b0f9d5b72549 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:11.662821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f577a97-aadd-49c6-87e3-b0f9d5b72549 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:11.690702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 140/288] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:11.706077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f577a97-aadd-49c6-87e3-b0f9d5b72549 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:11.852912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 149/289] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:11.859405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b14a2-121d-4f74-aa22-b2a3c254eec4 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:11.891267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f577a97-aadd-49c6-87e3-b0f9d5b72549 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:11.908396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:11.965987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1ca83-6311-4c34-abf2-32cac95edcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:11.966640 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1ca83-6311-4c34-abf2-32cac95edcad None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:11.966919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1ca83-6311-4c34-abf2-32cac95edcad None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:11.967188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1ca83-6311-4c34-abf2-32cac95edcad None admin] PATH_INFO: `/v3/users/b185426ee58b4d9ca878fab3369ddef7` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:11.969016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1ca83-6311-4c34-abf2-32cac95edcad None admin] RBAC: Authorizing `identity:get_user(user_id=b185426ee58b4d9ca878fab3369ddef7)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:11.970446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1ca83-6311-4c34-abf2-32cac95edcad None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:11.973186 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 141/290] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:11 2022] GET /identity/v3/users/b185426ee58b4d9ca878fab3369ddef7 => generated 297 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:11.979283 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1ca83-6311-4c34-abf2-32cac95edcad None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:12.000812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:12.066166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b14a2-121d-4f74-aa22-b2a3c254eec4 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:12.132776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd070b36-82a9-430f-a134-3a1b89e1d59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:12.134059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd070b36-82a9-430f-a134-3a1b89e1d59b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:12.134647 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd070b36-82a9-430f-a134-3a1b89e1d59b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:12.135817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd070b36-82a9-430f-a134-3a1b89e1d59b None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:12.137741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd070b36-82a9-430f-a134-3a1b89e1d59b None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:12.140166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd070b36-82a9-430f-a134-3a1b89e1d59b None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:12.144955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 150/291] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:11 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:12.156745 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd070b36-82a9-430f-a134-3a1b89e1d59b None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:12.215831 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1ca83-6311-4c34-abf2-32cac95edcad None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:12.281076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdb8c4-1b02-4c12-b1e6-c2d81c527dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:12.281725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdb8c4-1b02-4c12-b1e6-c2d81c527dc8 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:12.282006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdb8c4-1b02-4c12-b1e6-c2d81c527dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:12.282280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdb8c4-1b02-4c12-b1e6-c2d81c527dc8 None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:12.284176 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdb8c4-1b02-4c12-b1e6-c2d81c527dc8 None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:12.285808 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdb8c4-1b02-4c12-b1e6-c2d81c527dc8 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:12.293910 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 142/292] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:11 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:12.298885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdb8c4-1b02-4c12-b1e6-c2d81c527dc8 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:12.325169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:12.341204 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd070b36-82a9-430f-a134-3a1b89e1d59b None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:12.404009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152d91c-4a5d-407b-aaaf-4684b7389247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:12.404615 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152d91c-4a5d-407b-aaaf-4684b7389247 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:12.404908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152d91c-4a5d-407b-aaaf-4684b7389247 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:12.405219 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152d91c-4a5d-407b-aaaf-4684b7389247 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:12.406029 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152d91c-4a5d-407b-aaaf-4684b7389247 None admin] RBAC: Adding query filter params (role.id=f3b7ffe4d1f04af18f7da8d9a51031b3, scope.project.id=15984787df4448da8a1beaa8b7b15320, user.id=76c1816559674b8d849538aa34d3716b) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:12.406388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152d91c-4a5d-407b-aaaf-4684b7389247 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:12.407844 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152d91c-4a5d-407b-aaaf-4684b7389247 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:12.420075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 151/293] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:12 2022] GET /identity/v3/role_assignments?user.id=76c1816559674b8d849538aa34d3716b&scope.project.id=15984787df4448da8a1beaa8b7b15320&role.id=f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:12.437334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b4596-23b2-4dd9-880e-114428a7d9a7 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:12.438014 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b4596-23b2-4dd9-880e-114428a7d9a7 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:12.438497 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b4596-23b2-4dd9-880e-114428a7d9a7 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:12.440304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 152/294] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:37:12.441091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:12.451826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102fe92-595d-4500-a851-927b0240f320 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:12.452516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102fe92-595d-4500-a851-927b0240f320 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:12.453055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102fe92-595d-4500-a851-927b0240f320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:12.478327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-1102fe92-595d-4500-a851-927b0240f320 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:12.481304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1102fe92-595d-4500-a851-927b0240f320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:12.494282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdb8c4-1b02-4c12-b1e6-c2d81c527dc8 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:12.538434 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81941d-11fb-453d-bed0-a32b637b7f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:12.539203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81941d-11fb-453d-bed0-a32b637b7f8b None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:12.539517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81941d-11fb-453d-bed0-a32b637b7f8b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:12.539827 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81941d-11fb-453d-bed0-a32b637b7f8b None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320/users/b185426ee58b4d9ca878fab3369ddef7/roles/5f5e6a876c4a4fb0869e5bea07e7a65c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:12.542163 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81941d-11fb-453d-bed0-a32b637b7f8b None admin] RBAC: Authorizing `identity:create_grant(project_id=15984787df4448da8a1beaa8b7b15320, user_id=b185426ee58b4d9ca878fab3369ddef7, role_id=5f5e6a876c4a4fb0869e5bea07e7a65c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:12.544592 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81941d-11fb-453d-bed0-a32b637b7f8b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:12.560272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 143/295] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:37:12 2022] PUT /identity/v3/projects/15984787df4448da8a1beaa8b7b15320/users/b185426ee58b4d9ca878fab3369ddef7/roles/5f5e6a876c4a4fb0869e5bea07e7a65c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:37:12.561607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d6d38-badd-40d6-95ce-4cb54f168a5f None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:12.561905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d6d38-badd-40d6-95ce-4cb54f168a5f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:12.562186 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d6d38-badd-40d6-95ce-4cb54f168a5f None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:12.568498 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 144/296] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:12.570079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e67d0-ae16-4f0c-866d-f7166e19dbd2 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:12.570371 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e67d0-ae16-4f0c-866d-f7166e19dbd2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:12.570706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e67d0-ae16-4f0c-866d-f7166e19dbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:12.583709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 153/297] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:12.594790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-625e67d0-ae16-4f0c-866d-f7166e19dbd2 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:12.596080 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-625e67d0-ae16-4f0c-866d-f7166e19dbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:12.612727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102fe92-595d-4500-a851-927b0240f320 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:12.681111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 145/298] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:12.707641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e67d0-ae16-4f0c-866d-f7166e19dbd2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:12.721670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:12.740820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102fe92-595d-4500-a851-927b0240f320 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:12.784667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79795fac-4f76-42d8-8b4c-944ff820b6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:12.785423 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79795fac-4f76-42d8-8b4c-944ff820b6ab None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:12.785978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79795fac-4f76-42d8-8b4c-944ff820b6ab None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:12.786136 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79795fac-4f76-42d8-8b4c-944ff820b6ab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:12.789983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:12.803361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79795fac-4f76-42d8-8b4c-944ff820b6ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-79795fac-4f76-42d8-8b4c-944ff820b6ab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:12.810039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:12.810999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:12.811801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:12.811801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:12.811801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:12.811801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:12.811801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:12.811801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:12.811801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:12.811801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:12.811801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:12.811801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:12.811801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:37:12.811801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:12.811801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 154/299] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:12.817462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79795fac-4f76-42d8-8b4c-944ff820b6ab None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:12.846142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e67d0-ae16-4f0c-866d-f7166e19dbd2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:12.883467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de26c3d-c796-4026-bcd7-e392364e88a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:12.884125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de26c3d-c796-4026-bcd7-e392364e88a3 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:12.884473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de26c3d-c796-4026-bcd7-e392364e88a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:12.885065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de26c3d-c796-4026-bcd7-e392364e88a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:12.885961 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de26c3d-c796-4026-bcd7-e392364e88a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:12.887408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de26c3d-c796-4026-bcd7-e392364e88a3 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:12.892739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 146/300] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:12.896840 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de26c3d-c796-4026-bcd7-e392364e88a3 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:12.964364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79795fac-4f76-42d8-8b4c-944ff820b6ab None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:13.009201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18313181-cd88-424f-889c-4a7fcd09245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:13.010969 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18313181-cd88-424f-889c-4a7fcd09245c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:13.011566 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18313181-cd88-424f-889c-4a7fcd09245c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:13.012227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18313181-cd88-424f-889c-4a7fcd09245c None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:13.013542 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18313181-cd88-424f-889c-4a7fcd09245c None admin] RBAC: Adding query filter params (name=reader) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:13.013997 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18313181-cd88-424f-889c-4a7fcd09245c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:13.015641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18313181-cd88-424f-889c-4a7fcd09245c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:13.028704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 155/301] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:13.034192 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18313181-cd88-424f-889c-4a7fcd09245c None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:13.034462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de26c3d-c796-4026-bcd7-e392364e88a3 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:13.049699 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:13.081652 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330fe4a-771f-421f-be7c-bb2419a30882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:13.082304 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330fe4a-771f-421f-be7c-bb2419a30882 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:13.082607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330fe4a-771f-421f-be7c-bb2419a30882 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:13.083851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330fe4a-771f-421f-be7c-bb2419a30882 None admin] PATH_INFO: `/v3/groups` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:13.084559 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330fe4a-771f-421f-be7c-bb2419a30882 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:13.086060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330fe4a-771f-421f-be7c-bb2419a30882 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:13.091910 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:13.111096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 147/302] 10.222.0.69 () {60 vars in 1134 bytes} [Wed Aug 3 18:37:12 2022] POST /identity/v3/groups => generated 229 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:37:13.148947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27aecfc-e470-408f-99b9-05a002e86df6 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:13.149238 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27aecfc-e470-408f-99b9-05a002e86df6 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:13.149527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27aecfc-e470-408f-99b9-05a002e86df6 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:13.151728 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 148/303] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:37:13.157867 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e74aca-8e2b-4948-972c-f8f32907fe49 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:13.157867 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e74aca-8e2b-4948-972c-f8f32907fe49 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:13.158166 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e74aca-8e2b-4948-972c-f8f32907fe49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:13.174088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18313181-cd88-424f-889c-4a7fcd09245c None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:13.177481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-81e74aca-8e2b-4948-972c-f8f32907fe49 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:13.179240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-81e74aca-8e2b-4948-972c-f8f32907fe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:13.223414 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2cd78-f671-41bd-914f-20f6e296cf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:13.224083 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2cd78-f671-41bd-914f-20f6e296cf65 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:13.224415 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2cd78-f671-41bd-914f-20f6e296cf65 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:13.224700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2cd78-f671-41bd-914f-20f6e296cf65 None admin] PATH_INFO: `/v3/users/b008c7a092f14685ae026c8495b7966c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:13.244825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2cd78-f671-41bd-914f-20f6e296cf65 None admin] RBAC: Authorizing `identity:get_user(user_id=b008c7a092f14685ae026c8495b7966c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:13.246608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2cd78-f671-41bd-914f-20f6e296cf65 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:13.249835 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 156/304] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:13 2022] GET /identity/v3/users/b008c7a092f14685ae026c8495b7966c => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:13.255412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2cd78-f671-41bd-914f-20f6e296cf65 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:13.268175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 149/305] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:13.293458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e74aca-8e2b-4948-972c-f8f32907fe49 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:13.347063 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:13.397923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2cd78-f671-41bd-914f-20f6e296cf65 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:13.422931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e74aca-8e2b-4948-972c-f8f32907fe49 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:13.449461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:13.451423 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b329045-bf8a-44c1-b752-a8c01f136237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:13.452472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b329045-bf8a-44c1-b752-a8c01f136237 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:13.453407 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b329045-bf8a-44c1-b752-a8c01f136237 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:13.454025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b329045-bf8a-44c1-b752-a8c01f136237 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:13.455168 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b329045-bf8a-44c1-b752-a8c01f136237 None admin] RBAC: Adding query filter params (role.id=5f5e6a876c4a4fb0869e5bea07e7a65c, user.id=b008c7a092f14685ae026c8495b7966c, scope.system=all) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:13.455679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b329045-bf8a-44c1-b752-a8c01f136237 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:13.458171 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b329045-bf8a-44c1-b752-a8c01f136237 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:13.462580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c284e-c266-4d46-826c-60c870aa7c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:13.463767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c284e-c266-4d46-826c-60c870aa7c00 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:13.464056 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c284e-c266-4d46-826c-60c870aa7c00 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:13.464366 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c284e-c266-4d46-826c-60c870aa7c00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:13.466071 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 157/306] 10.222.0.69 () {56 vars in 1308 bytes} [Wed Aug 3 18:37:13 2022] GET /identity/v3/role_assignments?user.id=b008c7a092f14685ae026c8495b7966c&scope.system=all&role.id=5f5e6a876c4a4fb0869e5bea07e7a65c => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:13.481033 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c284e-c266-4d46-826c-60c870aa7c00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-7e9c284e-c266-4d46-826c-60c870aa7c00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:13.486554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:13.487726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:13.488567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:13.488567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:13.488567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:13.488567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:13.488567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:13.488567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:13.488567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:13.488567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:13.488567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:13.488567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:13.488567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:13.488567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:13.492465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 150/307] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:13.494476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b329045-bf8a-44c1-b752-a8c01f136237 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:13.613106 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b329045-bf8a-44c1-b752-a8c01f136237 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:13.648975 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:13.656590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f30c47-39ff-4ee0-bc2d-ce33511502bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:13.657500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f30c47-39ff-4ee0-bc2d-ce33511502bd None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:13.657983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f30c47-39ff-4ee0-bc2d-ce33511502bd None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:13.658480 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f30c47-39ff-4ee0-bc2d-ce33511502bd None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:13.659434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f30c47-39ff-4ee0-bc2d-ce33511502bd None admin] RBAC: Adding query filter params (name=member) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:13.659938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f30c47-39ff-4ee0-bc2d-ce33511502bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:13.661542 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f30c47-39ff-4ee0-bc2d-ce33511502bd None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:13.672130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 158/308] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:13.677498 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c284e-c266-4d46-826c-60c870aa7c00 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:13.787271 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c284e-c266-4d46-826c-60c870aa7c00 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:13.824724 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa104ece-bed1-4b9b-96fd-1f14e9acee37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:13.825378 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa104ece-bed1-4b9b-96fd-1f14e9acee37 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:13.825666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa104ece-bed1-4b9b-96fd-1f14e9acee37 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:13.825943 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa104ece-bed1-4b9b-96fd-1f14e9acee37 None admin] PATH_INFO: `/v3/users/a4740c15087f4f45895b50da687cb884` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:13.827130 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa104ece-bed1-4b9b-96fd-1f14e9acee37 None admin] RBAC: Authorizing `identity:get_user(user_id=a4740c15087f4f45895b50da687cb884)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:13.828610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa104ece-bed1-4b9b-96fd-1f14e9acee37 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:13.834893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 151/309] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:13 2022] GET /identity/v3/users/a4740c15087f4f45895b50da687cb884 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:13.836643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f30c47-39ff-4ee0-bc2d-ce33511502bd None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:13.950346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f30c47-39ff-4ee0-bc2d-ce33511502bd None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:13.985842 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aaacab-014b-4a1c-a49d-458fdcdd0115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:13.986515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aaacab-014b-4a1c-a49d-458fdcdd0115 None admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:13.986797 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aaacab-014b-4a1c-a49d-458fdcdd0115 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:13.987069 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aaacab-014b-4a1c-a49d-458fdcdd0115 None admin] PATH_INFO: `/v3/system/users/a4740c15087f4f45895b50da687cb884/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:13.988569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aaacab-014b-4a1c-a49d-458fdcdd0115 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a4740c15087f4f45895b50da687cb884, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:13.990062 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aaacab-014b-4a1c-a49d-458fdcdd0115 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:13.999550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 159/310] 10.222.0.69 () {58 vars in 1334 bytes} [Wed Aug 3 18:37:13 2022] PUT /identity/v3/system/users/a4740c15087f4f45895b50da687cb884/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:37:14.810433 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e41eca-6280-40a7-ace6-826a29b08dc2 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:14.810985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e41eca-6280-40a7-ace6-826a29b08dc2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:14.810985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e41eca-6280-40a7-ace6-826a29b08dc2 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:14.812505 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 152/311] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:14.817563 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289e63a-0761-4b3d-9ea3-ec69d81d9a9c None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:14.817951 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289e63a-0761-4b3d-9ea3-ec69d81d9a9c None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:14.818224 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289e63a-0761-4b3d-9ea3-ec69d81d9a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:14.836818 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-5289e63a-0761-4b3d-9ea3-ec69d81d9a9c None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:14.838939 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5289e63a-0761-4b3d-9ea3-ec69d81d9a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:14.891732 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:14.909947 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 160/312] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:14.943551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e41eca-6280-40a7-ace6-826a29b08dc2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:15.000458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:15.051524 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e41eca-6280-40a7-ace6-826a29b08dc2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:15.088821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236b3a0-db90-48c5-a3a4-5902e75a501d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:15.089601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236b3a0-db90-48c5-a3a4-5902e75a501d None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:15.089884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236b3a0-db90-48c5-a3a4-5902e75a501d None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:15.090156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236b3a0-db90-48c5-a3a4-5902e75a501d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:15.100864 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236b3a0-db90-48c5-a3a4-5902e75a501d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-b236b3a0-db90-48c5-a3a4-5902e75a501d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:15.106777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:15.107694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:15.108521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:15.108521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:15.108521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:15.108521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:15.108521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:15.108521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:15.108521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:15.108521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:15.108521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:15.108521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:15.108521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:37:15.108521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:15.108521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 153/313] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:15.117557 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289e63a-0761-4b3d-9ea3-ec69d81d9a9c None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:15.229610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289e63a-0761-4b3d-9ea3-ec69d81d9a9c None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:15.266456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8657fa-d8d3-4b61-99cb-7843894b85a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:15.267433 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8657fa-d8d3-4b61-99cb-7843894b85a8 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:15.267911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8657fa-d8d3-4b61-99cb-7843894b85a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:15.268451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8657fa-d8d3-4b61-99cb-7843894b85a8 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:15.269377 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8657fa-d8d3-4b61-99cb-7843894b85a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:15.269878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8657fa-d8d3-4b61-99cb-7843894b85a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:15.271451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8657fa-d8d3-4b61-99cb-7843894b85a8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:15.280957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 161/314] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:15.288312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236b3a0-db90-48c5-a3a4-5902e75a501d None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:15.386691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad033a0-1681-432c-bf18-4a676504c16b None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:15.386931 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad033a0-1681-432c-bf18-4a676504c16b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:15.387176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad033a0-1681-432c-bf18-4a676504c16b None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:15.388874 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 162/315] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:15.394127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfe905-912d-49a7-8a27-31cf1f7c406e None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:15.394417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfe905-912d-49a7-8a27-31cf1f7c406e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:15.394700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfe905-912d-49a7-8a27-31cf1f7c406e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:15.399288 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236b3a0-db90-48c5-a3a4-5902e75a501d None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:15.414087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:15.415305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-5fbfe905-912d-49a7-8a27-31cf1f7c406e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:15.417595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5fbfe905-912d-49a7-8a27-31cf1f7c406e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:15.438792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb0c0a-4464-4cde-95f1-22afee4ec098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:15.439441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb0c0a-4464-4cde-95f1-22afee4ec098 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:15.439725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb0c0a-4464-4cde-95f1-22afee4ec098 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:15.439999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb0c0a-4464-4cde-95f1-22afee4ec098 None admin] PATH_INFO: `/v3/users/b185426ee58b4d9ca878fab3369ddef7` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:15.441224 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb0c0a-4464-4cde-95f1-22afee4ec098 None admin] RBAC: Authorizing `identity:get_user(user_id=b185426ee58b4d9ca878fab3369ddef7)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:15.442655 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb0c0a-4464-4cde-95f1-22afee4ec098 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:15.444598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 154/316] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:15 2022] GET /identity/v3/users/b185426ee58b4d9ca878fab3369ddef7 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:15.450474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb0c0a-4464-4cde-95f1-22afee4ec098 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:15.472461 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:15.515860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 163/317] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:15.515860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8464693-9af4-443d-a919-2431afefa2b0 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:15.515860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8464693-9af4-443d-a919-2431afefa2b0 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:15.515860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8464693-9af4-443d-a919-2431afefa2b0 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:15.515860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 164/318] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:15.520466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee1f7b-07eb-4c76-bd08-af1c287f49eb None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:15.520752 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee1f7b-07eb-4c76-bd08-af1c287f49eb None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:15.521024 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee1f7b-07eb-4c76-bd08-af1c287f49eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:15.540442 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-c5ee1f7b-07eb-4c76-bd08-af1c287f49eb None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:15.542048 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c5ee1f7b-07eb-4c76-bd08-af1c287f49eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:15.571797 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb0c0a-4464-4cde-95f1-22afee4ec098 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:15.609851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a268d-2884-49f7-bc3d-c57bee0b0477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:15.610617 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a268d-2884-49f7-bc3d-c57bee0b0477 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:15.611012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a268d-2884-49f7-bc3d-c57bee0b0477 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:15.611334 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a268d-2884-49f7-bc3d-c57bee0b0477 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:15.619027 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 165/319] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:15.619801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee1f7b-07eb-4c76-bd08-af1c287f49eb None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:15.669554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:15.690421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a268d-2884-49f7-bc3d-c57bee0b0477 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:15.691890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a268d-2884-49f7-bc3d-c57bee0b0477 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:15.741449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee1f7b-07eb-4c76-bd08-af1c287f49eb None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:15.752721 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:15.787833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671b813-fb5b-4936-8a12-7dcfb958aefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:15.788885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671b813-fb5b-4936-8a12-7dcfb958aefb None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:15.789369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671b813-fb5b-4936-8a12-7dcfb958aefb None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:15.789656 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671b813-fb5b-4936-8a12-7dcfb958aefb None admin] PATH_INFO: `/v3/domains/default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:15.790877 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3671b813-fb5b-4936-8a12-7dcfb958aefb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:15.792426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3671b813-fb5b-4936-8a12-7dcfb958aefb None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:15.794056 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 166/320] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:15.794833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671b813-fb5b-4936-8a12-7dcfb958aefb None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:15.803025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 155/321] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:15.805203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a268d-2884-49f7-bc3d-c57bee0b0477 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:15.895579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:15.904633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671b813-fb5b-4936-8a12-7dcfb958aefb None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:15.919865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a268d-2884-49f7-bc3d-c57bee0b0477 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:15.942415 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83240424-bc6f-481f-8dfe-433bb3d4179d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:15.943140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83240424-bc6f-481f-8dfe-433bb3d4179d None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:15.943428 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83240424-bc6f-481f-8dfe-433bb3d4179d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:15.943702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83240424-bc6f-481f-8dfe-433bb3d4179d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:15.955865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83240424-bc6f-481f-8dfe-433bb3d4179d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:15.958646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827d404-f309-4820-b70b-6ea41ced4257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:15.959276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827d404-f309-4820-b70b-6ea41ced4257 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:15.959554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827d404-f309-4820-b70b-6ea41ced4257 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:15.959823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827d404-f309-4820-b70b-6ea41ced4257 None admin] PATH_INFO: `/v3/groups` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:15.960514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827d404-f309-4820-b70b-6ea41ced4257 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-83240424-bc6f-481f-8dfe-433bb3d4179d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:15.961305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:15.962165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:15.963041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:15.963041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:15.963041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:15.963041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:15.963041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:15.963041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:15.963041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:15.963041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:15.963041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:15.963041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:15.963041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:37:15.963041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:15.963660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827d404-f309-4820-b70b-6ea41ced4257 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:15.963728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 167/322] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:15.963728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83240424-bc6f-481f-8dfe-433bb3d4179d None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:15.984984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 156/323] 10.222.0.69 () {60 vars in 1134 bytes} [Wed Aug 3 18:37:15 2022] POST /identity/v3/groups => generated 226 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:37:15.985631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827d404-f309-4820-b70b-6ea41ced4257 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:16.044371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:16.126051 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83240424-bc6f-481f-8dfe-433bb3d4179d None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:16.144165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827d404-f309-4820-b70b-6ea41ced4257 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:16.166026 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cd2ed-d356-4d1f-9451-19fe29234c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:16.166994 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cd2ed-d356-4d1f-9451-19fe29234c2f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:16.167732 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cd2ed-d356-4d1f-9451-19fe29234c2f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:16.168013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cd2ed-d356-4d1f-9451-19fe29234c2f None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:16.169318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cd2ed-d356-4d1f-9451-19fe29234c2f None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:16.171144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cd2ed-d356-4d1f-9451-19fe29234c2f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:16.173011 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 168/324] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:15 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:16.174220 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe72c7-d504-4acc-9e20-e16f04798048 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:16.174441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe72c7-d504-4acc-9e20-e16f04798048 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:16.174709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe72c7-d504-4acc-9e20-e16f04798048 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:16.176899 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 169/325] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:16.181418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe72c7-d504-4acc-9e20-e16f04798048 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:16.187851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91684e03-8e91-469e-8ae3-2a01ba01c2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:16.188507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91684e03-8e91-469e-8ae3-2a01ba01c2d9 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:16.188788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91684e03-8e91-469e-8ae3-2a01ba01c2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:16.189060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91684e03-8e91-469e-8ae3-2a01ba01c2d9 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:16.190097 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91684e03-8e91-469e-8ae3-2a01ba01c2d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:16.190097 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91684e03-8e91-469e-8ae3-2a01ba01c2d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:16.191436 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91684e03-8e91-469e-8ae3-2a01ba01c2d9 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:16.203271 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 157/326] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:16.209073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52ebf2-acca-4e45-b3eb-2c783633bc0a None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:16.209073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52ebf2-acca-4e45-b3eb-2c783633bc0a None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:16.209310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52ebf2-acca-4e45-b3eb-2c783633bc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:16.217023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:16.230546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-4f52ebf2-acca-4e45-b3eb-2c783633bc0a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:16.233527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4f52ebf2-acca-4e45-b3eb-2c783633bc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:16.319109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe72c7-d504-4acc-9e20-e16f04798048 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:16.319588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 158/327] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:16.324399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52ebf2-acca-4e45-b3eb-2c783633bc0a None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:16.324594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:16.365168 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25f742-a0a7-426d-9399-d509bbdaae25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:16.365800 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25f742-a0a7-426d-9399-d509bbdaae25 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:16.366078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25f742-a0a7-426d-9399-d509bbdaae25 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:16.366349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25f742-a0a7-426d-9399-d509bbdaae25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:16.367156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25f742-a0a7-426d-9399-d509bbdaae25 None admin] RBAC: Adding query filter params (role.id=5f5e6a876c4a4fb0869e5bea07e7a65c, scope.project.id=15984787df4448da8a1beaa8b7b15320, user.id=b185426ee58b4d9ca878fab3369ddef7) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:16.367466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25f742-a0a7-426d-9399-d509bbdaae25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:16.370492 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25f742-a0a7-426d-9399-d509bbdaae25 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:16.378149 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 170/328] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:16 2022] GET /identity/v3/role_assignments?user.id=b185426ee58b4d9ca878fab3369ddef7&scope.project.id=15984787df4448da8a1beaa8b7b15320&role.id=5f5e6a876c4a4fb0869e5bea07e7a65c => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:16.379230 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25f742-a0a7-426d-9399-d509bbdaae25 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:16.500409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52ebf2-acca-4e45-b3eb-2c783633bc0a None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:16.528275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25f742-a0a7-426d-9399-d509bbdaae25 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:16.543531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:16.548831 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d94a7-a604-4f5b-8f55-9ef94ed33831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:16.549495 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d94a7-a604-4f5b-8f55-9ef94ed33831 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:16.549881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d94a7-a604-4f5b-8f55-9ef94ed33831 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:16.550054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d94a7-a604-4f5b-8f55-9ef94ed33831 None admin] PATH_INFO: `/v3/users/b008c7a092f14685ae026c8495b7966c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:16.551599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d94a7-a604-4f5b-8f55-9ef94ed33831 None admin] RBAC: Authorizing `identity:get_user(user_id=b008c7a092f14685ae026c8495b7966c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:16.552798 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d94a7-a604-4f5b-8f55-9ef94ed33831 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:16.554491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 159/329] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:16 2022] GET /identity/v3/users/b008c7a092f14685ae026c8495b7966c => generated 293 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:16.565402 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d94a7-a604-4f5b-8f55-9ef94ed33831 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:16.582121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03c5ef-cebe-4e46-8c45-56778b5f5db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:16.583010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c5ef-cebe-4e46-8c45-56778b5f5db0 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:16.583449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c5ef-cebe-4e46-8c45-56778b5f5db0 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:16.583982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c5ef-cebe-4e46-8c45-56778b5f5db0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:16.596030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03c5ef-cebe-4e46-8c45-56778b5f5db0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-8b03c5ef-cebe-4e46-8c45-56778b5f5db0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:16.602194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:16.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:16.604132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:16.604132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:16.604132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:16.604132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:16.604132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:16.604132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:16.604132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:16.604132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:16.604132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:16.604132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:16.604132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:16.604132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:16.606476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 171/330] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:16.612556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03c5ef-cebe-4e46-8c45-56778b5f5db0 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:16.657949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:16.762248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d94a7-a604-4f5b-8f55-9ef94ed33831 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:16.762626 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03c5ef-cebe-4e46-8c45-56778b5f5db0 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:16.817460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04cd6b-3fca-4ffe-8839-f088a21a1d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:16.818096 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04cd6b-3fca-4ffe-8839-f088a21a1d4d None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:16.818367 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04cd6b-3fca-4ffe-8839-f088a21a1d4d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:16.818638 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04cd6b-3fca-4ffe-8839-f088a21a1d4d None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:16.819361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04cd6b-3fca-4ffe-8839-f088a21a1d4d None admin] RBAC: Adding query filter params (name=member) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:16.819662 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04cd6b-3fca-4ffe-8839-f088a21a1d4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:16.820933 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd932d-a09a-45c7-9a1d-616bbe2972d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:16.821062 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04cd6b-3fca-4ffe-8839-f088a21a1d4d None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:16.822224 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd932d-a09a-45c7-9a1d-616bbe2972d4 None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:16.823344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd932d-a09a-45c7-9a1d-616bbe2972d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:16.825090 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd932d-a09a-45c7-9a1d-616bbe2972d4 None admin] PATH_INFO: `/v3/system/users/b008c7a092f14685ae026c8495b7966c/roles/5f5e6a876c4a4fb0869e5bea07e7a65c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:16.827355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd932d-a09a-45c7-9a1d-616bbe2972d4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b008c7a092f14685ae026c8495b7966c, role_id=5f5e6a876c4a4fb0869e5bea07e7a65c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:16.829007 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd932d-a09a-45c7-9a1d-616bbe2972d4 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:16.835972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 172/331] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:16.842728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04cd6b-3fca-4ffe-8839-f088a21a1d4d None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:16.843965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 160/332] 10.222.0.69 () {58 vars in 1334 bytes} [Wed Aug 3 18:37:16 2022] PUT /identity/v3/system/users/b008c7a092f14685ae026c8495b7966c/roles/5f5e6a876c4a4fb0869e5bea07e7a65c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:37:16.977416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:16.992505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04cd6b-3fca-4ffe-8839-f088a21a1d4d None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:17.039151 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70e9c0-4b8c-4811-b552-afd0f16f84b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:17.039746 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70e9c0-4b8c-4811-b552-afd0f16f84b2 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:17.040025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70e9c0-4b8c-4811-b552-afd0f16f84b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:17.040312 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70e9c0-4b8c-4811-b552-afd0f16f84b2 None admin] PATH_INFO: `/v3/users/a4740c15087f4f45895b50da687cb884` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:17.045208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70e9c0-4b8c-4811-b552-afd0f16f84b2 None admin] RBAC: Authorizing `identity:get_user(user_id=a4740c15087f4f45895b50da687cb884)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:17.046736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70e9c0-4b8c-4811-b552-afd0f16f84b2 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:17.048507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 173/333] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:16 2022] GET /identity/v3/users/a4740c15087f4f45895b50da687cb884 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:17.055262 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd932d-a09a-45c7-9a1d-616bbe2972d4 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:17.143066 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:17.179627 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd932d-a09a-45c7-9a1d-616bbe2972d4 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:17.219130 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7867f0-1f72-42ff-aa65-e0a94f92daef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:17.219761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7867f0-1f72-42ff-aa65-e0a94f92daef None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:17.220044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7867f0-1f72-42ff-aa65-e0a94f92daef None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:17.220367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7867f0-1f72-42ff-aa65-e0a94f92daef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:17.221180 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7867f0-1f72-42ff-aa65-e0a94f92daef None admin] RBAC: Adding query filter params (role.id=f3b7ffe4d1f04af18f7da8d9a51031b3, user.id=a4740c15087f4f45895b50da687cb884, scope.system=all) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:17.221493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7867f0-1f72-42ff-aa65-e0a94f92daef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:17.222935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7867f0-1f72-42ff-aa65-e0a94f92daef None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:17.238443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 161/334] 10.222.0.69 () {56 vars in 1308 bytes} [Wed Aug 3 18:37:17 2022] GET /identity/v3/role_assignments?user.id=a4740c15087f4f45895b50da687cb884&scope.system=all&role.id=f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:18.685793 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ca6e8-5fec-4323-8cbf-c7ddc743e21e None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:18.687351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ca6e8-5fec-4323-8cbf-c7ddc743e21e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:18.687351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ca6e8-5fec-4323-8cbf-c7ddc743e21e None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:18.689096 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 174/335] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:18.696527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587d960-c986-4258-8e80-7b9548349541 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:18.698219 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587d960-c986-4258-8e80-7b9548349541 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:18.698720 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587d960-c986-4258-8e80-7b9548349541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:18.719014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-c587d960-c986-4258-8e80-7b9548349541 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:18.720664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c587d960-c986-4258-8e80-7b9548349541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:18.801635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 162/336] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:18.825893 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ca6e8-5fec-4323-8cbf-c7ddc743e21e None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:18.954091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ca6e8-5fec-4323-8cbf-c7ddc743e21e None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:18.991310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe3cd2-049f-4ef5-837d-1ad074c828d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:18.992418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe3cd2-049f-4ef5-837d-1ad074c828d6 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:18.992891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe3cd2-049f-4ef5-837d-1ad074c828d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:18.993352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe3cd2-049f-4ef5-837d-1ad074c828d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:19.029839 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe3cd2-049f-4ef5-837d-1ad074c828d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-1afe3cd2-049f-4ef5-837d-1ad074c828d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:19.034838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:19.035749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:19.036581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:19.036581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:19.036581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:19.036581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:19.036581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:19.036581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:19.036581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:19.036581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:19.036581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:19.036581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:19.036581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:19.036581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:19.039209 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 175/337] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:19.049378 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587d960-c986-4258-8e80-7b9548349541 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:19.152748 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:19.225480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587d960-c986-4258-8e80-7b9548349541 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:19.277862 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d575faf-e691-4568-8dcf-5d2546bc1ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:19.278506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d575faf-e691-4568-8dcf-5d2546bc1ee6 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:19.278793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d575faf-e691-4568-8dcf-5d2546bc1ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:19.279074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d575faf-e691-4568-8dcf-5d2546bc1ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:19.279897 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d575faf-e691-4568-8dcf-5d2546bc1ee6 None admin] RBAC: Adding query filter params (name=member) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:19.280218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d575faf-e691-4568-8dcf-5d2546bc1ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:19.281638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d575faf-e691-4568-8dcf-5d2546bc1ee6 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:19.295719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 163/338] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:19.305393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe3cd2-049f-4ef5-837d-1ad074c828d6 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:19.341078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:19.387735 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c09887-62fa-4e2a-8c25-58b4ccf03452 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:19.388434 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c09887-62fa-4e2a-8c25-58b4ccf03452 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:19.388937 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c09887-62fa-4e2a-8c25-58b4ccf03452 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:19.390774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 164/339] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:19.397932 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c54ff-63c3-4803-a65c-53f443d672f5 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:19.398504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c54ff-63c3-4803-a65c-53f443d672f5 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:19.399048 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c54ff-63c3-4803-a65c-53f443d672f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:19.428037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-4c2c54ff-63c3-4803-a65c-53f443d672f5 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:19.430644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4c2c54ff-63c3-4803-a65c-53f443d672f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:19.473847 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe3cd2-049f-4ef5-837d-1ad074c828d6 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:19.558446 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a323cb9-0991-4d87-bdf5-2e4da52480c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:19.558446 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a323cb9-0991-4d87-bdf5-2e4da52480c6 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:19.558446 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a323cb9-0991-4d87-bdf5-2e4da52480c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:19.558446 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a323cb9-0991-4d87-bdf5-2e4da52480c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:19.575065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 165/340] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:19.576820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb56c-4556-4cbc-9697-a57799903cbc None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:19.577415 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb56c-4556-4cbc-9697-a57799903cbc None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:19.577950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb56c-4556-4cbc-9697-a57799903cbc None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:19.579915 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 166/341] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:19.585874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74348994-a41d-4e4b-a578-e287b4ab1096 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:19.586483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74348994-a41d-4e4b-a578-e287b4ab1096 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:19.587000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74348994-a41d-4e4b-a578-e287b4ab1096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:19.605802 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-74348994-a41d-4e4b-a578-e287b4ab1096 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:19.607342 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-74348994-a41d-4e4b-a578-e287b4ab1096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:19.658699 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:19.686807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a323cb9-0991-4d87-bdf5-2e4da52480c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:19.688309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a323cb9-0991-4d87-bdf5-2e4da52480c6 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:19.692175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 167/342] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:19.693316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74348994-a41d-4e4b-a578-e287b4ab1096 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:19.721283 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:19.804379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74348994-a41d-4e4b-a578-e287b4ab1096 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:19.844200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d9c87-9771-49d6-8310-9b382efa1e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:19.845219 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d9c87-9771-49d6-8310-9b382efa1e24 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:19.845714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d9c87-9771-49d6-8310-9b382efa1e24 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:19.846222 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d9c87-9771-49d6-8310-9b382efa1e24 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:19.858786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d9c87-9771-49d6-8310-9b382efa1e24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-e43d9c87-9771-49d6-8310-9b382efa1e24 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:19.864220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:19.865150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:19.866014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:19.866014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:19.866014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:19.866014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:19.866014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:19.866014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:19.866014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:19.866014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:19.866014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:19.866014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:19.866014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:19.866014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:19.867608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 168/343] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:37:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:19.868607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d9c87-9771-49d6-8310-9b382efa1e24 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:19.888038 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 176/344] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:19.890145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529904b-65e3-40b9-8eae-7167db559ce3 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:19.890676 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529904b-65e3-40b9-8eae-7167db559ce3 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:19.891152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529904b-65e3-40b9-8eae-7167db559ce3 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:19.892970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 177/345] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:19.893863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529904b-65e3-40b9-8eae-7167db559ce3 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:19.924114 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:20.002870 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d9c87-9771-49d6-8310-9b382efa1e24 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:20.018957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529904b-65e3-40b9-8eae-7167db559ce3 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:20.037504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:20.044120 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366c756-b67b-4ec3-b57a-2ba6080d3315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:20.045032 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366c756-b67b-4ec3-b57a-2ba6080d3315 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:20.045321 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366c756-b67b-4ec3-b57a-2ba6080d3315 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:20.045601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366c756-b67b-4ec3-b57a-2ba6080d3315 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:20.058220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366c756-b67b-4ec3-b57a-2ba6080d3315 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:20.059004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ed80c-ca3d-4b3d-972d-95b03491b9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:20.059642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ed80c-ca3d-4b3d-972d-95b03491b9aa None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:20.059927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ed80c-ca3d-4b3d-972d-95b03491b9aa None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:20.060209 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ed80c-ca3d-4b3d-972d-95b03491b9aa None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:20.060985 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ed80c-ca3d-4b3d-972d-95b03491b9aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:20.061293 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ed80c-ca3d-4b3d-972d-95b03491b9aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:20.062759 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ed80c-ca3d-4b3d-972d-95b03491b9aa None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-0366c756-b67b-4ec3-b57a-2ba6080d3315 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:20.063363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:20.064275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:20.065136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:20.065136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:20.065136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:20.065136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:20.065136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:20.065136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:20.065136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:20.065136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:20.065136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:20.065136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:20.065136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:20.065136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:20.065136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 169/346] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:20.065896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366c756-b67b-4ec3-b57a-2ba6080d3315 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:20.073034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 178/347] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:20.074362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217b2c9-3cd4-482e-aae4-6f1edb8f3014 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:20.074653 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217b2c9-3cd4-482e-aae4-6f1edb8f3014 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:20.074934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217b2c9-3cd4-482e-aae4-6f1edb8f3014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:20.093212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-7217b2c9-3cd4-482e-aae4-6f1edb8f3014 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:20.094486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7217b2c9-3cd4-482e-aae4-6f1edb8f3014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:20.170594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 179/348] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:20.171458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217b2c9-3cd4-482e-aae4-6f1edb8f3014 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:20.190139 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:20.204688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366c756-b67b-4ec3-b57a-2ba6080d3315 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:20.251635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16cd5a-bbda-4080-b8b1-d4e594138ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:20.252316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16cd5a-bbda-4080-b8b1-d4e594138ede None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:20.252599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16cd5a-bbda-4080-b8b1-d4e594138ede None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:20.252879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16cd5a-bbda-4080-b8b1-d4e594138ede None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:20.254367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16cd5a-bbda-4080-b8b1-d4e594138ede None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:20.255919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16cd5a-bbda-4080-b8b1-d4e594138ede None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:20.257782 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 170/349] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:20 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:20.258671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16cd5a-bbda-4080-b8b1-d4e594138ede None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:20.285682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217b2c9-3cd4-482e-aae4-6f1edb8f3014 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:20.298493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:20.326506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b04ae6-fe5a-49b0-8ce0-6181db98ab7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:20.327149 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b04ae6-fe5a-49b0-8ce0-6181db98ab7b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:20.327535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b04ae6-fe5a-49b0-8ce0-6181db98ab7b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:20.327972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b04ae6-fe5a-49b0-8ce0-6181db98ab7b None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:20.328873 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b04ae6-fe5a-49b0-8ce0-6181db98ab7b None admin] RBAC: Adding query filter params (name=member) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:20.329189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b04ae6-fe5a-49b0-8ce0-6181db98ab7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:20.330669 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b04ae6-fe5a-49b0-8ce0-6181db98ab7b None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:20.340863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 180/350] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:20.341516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b04ae6-fe5a-49b0-8ce0-6181db98ab7b None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:20.370626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16cd5a-bbda-4080-b8b1-d4e594138ede None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:20.409377 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857a3c3-632b-44d0-be17-80dfd4e38b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:20.410015 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857a3c3-632b-44d0-be17-80dfd4e38b3e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:20.410411 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857a3c3-632b-44d0-be17-80dfd4e38b3e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:20.410698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857a3c3-632b-44d0-be17-80dfd4e38b3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:20.443585 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:20.456751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b04ae6-fe5a-49b0-8ce0-6181db98ab7b None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:20.487544 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857a3c3-632b-44d0-be17-80dfd4e38b3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:20.489161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857a3c3-632b-44d0-be17-80dfd4e38b3e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:20.496429 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d1c87-af11-419a-a8f3-af4de3d13ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:20.497186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d1c87-af11-419a-a8f3-af4de3d13ce1 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:20.497545 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d1c87-af11-419a-a8f3-af4de3d13ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:20.497831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d1c87-af11-419a-a8f3-af4de3d13ce1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:20.511745 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d1c87-af11-419a-a8f3-af4de3d13ce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-368d1c87-af11-419a-a8f3-af4de3d13ce1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:20.517217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:20.519802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:20.521915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:20.521915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:20.521915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:20.521915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:20.521915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:20.521915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:20.521915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:20.521915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:20.521915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:20.521915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:20.521915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 18:37:20.521915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:20.521915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 181/351] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:20.521915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d1c87-af11-419a-a8f3-af4de3d13ce1 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:20.556799 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:20.603266 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 171/352] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:20.604231 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857a3c3-632b-44d0-be17-80dfd4e38b3e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:20.641311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d1c87-af11-419a-a8f3-af4de3d13ce1 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:20.676368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:20.681839 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf480c-c547-48a2-bfdf-77d97fb93728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:20.683151 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf480c-c547-48a2-bfdf-77d97fb93728 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:20.683435 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf480c-c547-48a2-bfdf-77d97fb93728 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:20.683713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf480c-c547-48a2-bfdf-77d97fb93728 None admin] PATH_INFO: `/v3/groups/7315f7c7db09430a962b9b45c222cfd1` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:20.690665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf480c-c547-48a2-bfdf-77d97fb93728 None admin] RBAC: Authorizing `identity:get_group(group_id=7315f7c7db09430a962b9b45c222cfd1)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:20.692275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf480c-c547-48a2-bfdf-77d97fb93728 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:20.694523 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 182/353] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Aug 3 18:37:20 2022] GET /identity/v3/groups/7315f7c7db09430a962b9b45c222cfd1 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:20.695469 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf480c-c547-48a2-bfdf-77d97fb93728 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:20.721248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857a3c3-632b-44d0-be17-80dfd4e38b3e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:20.767911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82585f-a570-4efe-81bf-b08f2b72b441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:20.768586 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82585f-a570-4efe-81bf-b08f2b72b441 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:20.768926 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82585f-a570-4efe-81bf-b08f2b72b441 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:20.769149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82585f-a570-4efe-81bf-b08f2b72b441 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:20.816750 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:20.825354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf480c-c547-48a2-bfdf-77d97fb93728 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:20.862876 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82585f-a570-4efe-81bf-b08f2b72b441 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:20.866090 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82585f-a570-4efe-81bf-b08f2b72b441 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:20.872282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ebedb-c7a6-4beb-a2ae-272284bc090c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:20.872948 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ebedb-c7a6-4beb-a2ae-272284bc090c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:20.873232 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ebedb-c7a6-4beb-a2ae-272284bc090c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:20.873511 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ebedb-c7a6-4beb-a2ae-272284bc090c None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:20.874247 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ebedb-c7a6-4beb-a2ae-272284bc090c None admin] RBAC: Adding query filter params (name=reader) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:20.874554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ebedb-c7a6-4beb-a2ae-272284bc090c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:20.875928 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ebedb-c7a6-4beb-a2ae-272284bc090c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:20.886172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 183/354] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:20.891446 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ebedb-c7a6-4beb-a2ae-272284bc090c None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:20.945385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:21.012104 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 172/355] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:21.013274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82585f-a570-4efe-81bf-b08f2b72b441 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:21.029427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ebedb-c7a6-4beb-a2ae-272284bc090c None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:21.073610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5fb56-0cb7-4971-b30a-19d2637ef6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:21.074743 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5fb56-0cb7-4971-b30a-19d2637ef6d0 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:21.075145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5fb56-0cb7-4971-b30a-19d2637ef6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:21.075432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5fb56-0cb7-4971-b30a-19d2637ef6d0 None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:21.076707 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5fb56-0cb7-4971-b30a-19d2637ef6d0 None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:21.078297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5fb56-0cb7-4971-b30a-19d2637ef6d0 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:21.080940 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 184/356] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:20 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:21.082961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a9158-a083-4f18-affc-89f257b8ecbc None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:21.083504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a9158-a083-4f18-affc-89f257b8ecbc None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:21.084344 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a9158-a083-4f18-affc-89f257b8ecbc None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:21.086617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 185/357] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:21.087403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a9158-a083-4f18-affc-89f257b8ecbc None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:21.124324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:21.137602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82585f-a570-4efe-81bf-b08f2b72b441 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:21.184737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429a545-f557-4bdf-8a20-bfaea62dc09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:21.185350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429a545-f557-4bdf-8a20-bfaea62dc09c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:21.185653 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429a545-f557-4bdf-8a20-bfaea62dc09c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:21.185918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429a545-f557-4bdf-8a20-bfaea62dc09c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:21.186730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429a545-f557-4bdf-8a20-bfaea62dc09c None admin] RBAC: Adding query filter params (group.id=7315f7c7db09430a962b9b45c222cfd1, role.id=f3b7ffe4d1f04af18f7da8d9a51031b3, scope.project.id=42291678aa1842ff9db31b9d1e39dc2c) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:21.187041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429a545-f557-4bdf-8a20-bfaea62dc09c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:21.189306 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429a545-f557-4bdf-8a20-bfaea62dc09c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:21.195182 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 173/358] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Aug 3 18:37:21 2022] GET /identity/v3/role_assignments?group.id=7315f7c7db09430a962b9b45c222cfd1&scope.project.id=42291678aa1842ff9db31b9d1e39dc2c&role.id=f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:21.196039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429a545-f557-4bdf-8a20-bfaea62dc09c None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:21.235168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:21.236216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a9158-a083-4f18-affc-89f257b8ecbc None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:21.288518 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181251a7-fac0-4324-8cf8-0c75c28e623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:21.289182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181251a7-fac0-4324-8cf8-0c75c28e623a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:21.289465 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181251a7-fac0-4324-8cf8-0c75c28e623a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:21.289913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181251a7-fac0-4324-8cf8-0c75c28e623a None admin] PATH_INFO: `/v3/users/b008c7a092f14685ae026c8495b7966c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:21.294609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181251a7-fac0-4324-8cf8-0c75c28e623a None admin] RBAC: Authorizing `identity:get_user(user_id=b008c7a092f14685ae026c8495b7966c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:21.294609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181251a7-fac0-4324-8cf8-0c75c28e623a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:21.296687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 186/359] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:37:21 2022] GET /identity/v3/users/b008c7a092f14685ae026c8495b7966c => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:21.299060 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181251a7-fac0-4324-8cf8-0c75c28e623a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:21.343349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429a545-f557-4bdf-8a20-bfaea62dc09c None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:21.387342 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a384abc-8217-44bc-afea-25b113467cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:21.388882 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a384abc-8217-44bc-afea-25b113467cd3 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:21.389092 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a384abc-8217-44bc-afea-25b113467cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:21.389368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a384abc-8217-44bc-afea-25b113467cd3 None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:21.390737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a384abc-8217-44bc-afea-25b113467cd3 None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:21.392181 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a384abc-8217-44bc-afea-25b113467cd3 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:21.396316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 174/360] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:21 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:21.396316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4703a-7d13-40f9-9812-0edf5e5b0855 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:21.396316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4703a-7d13-40f9-9812-0edf5e5b0855 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:21.396316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4703a-7d13-40f9-9812-0edf5e5b0855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:21.417121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-84e4703a-7d13-40f9-9812-0edf5e5b0855 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:21.418579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-84e4703a-7d13-40f9-9812-0edf5e5b0855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:21.433091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:21.445888 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181251a7-fac0-4324-8cf8-0c75c28e623a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:21.490594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed2f39-d78f-4191-9d61-f3317e86c552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:21.491809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed2f39-d78f-4191-9d61-f3317e86c552 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:21.492452 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed2f39-d78f-4191-9d61-f3317e86c552 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:21.492737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed2f39-d78f-4191-9d61-f3317e86c552 None admin] PATH_INFO: `/v3/groups/7315f7c7db09430a962b9b45c222cfd1` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:21.494452 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed2f39-d78f-4191-9d61-f3317e86c552 None admin] RBAC: Authorizing `identity:get_group(group_id=7315f7c7db09430a962b9b45c222cfd1)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:21.496201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed2f39-d78f-4191-9d61-f3317e86c552 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:21.499199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 187/361] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Aug 3 18:37:21 2022] GET /identity/v3/groups/7315f7c7db09430a962b9b45c222cfd1 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:21.500488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afedac-858f-4083-8d59-b6ad85c500c0 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:21.500775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afedac-858f-4083-8d59-b6ad85c500c0 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:21.501050 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afedac-858f-4083-8d59-b6ad85c500c0 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:21.502296 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 188/362] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:21.503014 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afedac-858f-4083-8d59-b6ad85c500c0 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:21.504079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 175/363] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:21.505204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4703a-7d13-40f9-9812-0edf5e5b0855 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:21.543678 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:21.640641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4703a-7d13-40f9-9812-0edf5e5b0855 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:21.647803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afedac-858f-4083-8d59-b6ad85c500c0 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:21.683156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecc79f-1680-4d28-8f0e-f800cedff6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:21.683156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecc79f-1680-4d28-8f0e-f800cedff6bc None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:21.683156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecc79f-1680-4d28-8f0e-f800cedff6bc None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:21.683525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecc79f-1680-4d28-8f0e-f800cedff6bc None admin] PATH_INFO: `/v3/groups/7315f7c7db09430a962b9b45c222cfd1` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:21.684782 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecc79f-1680-4d28-8f0e-f800cedff6bc None admin] RBAC: Authorizing `identity:get_group(group_id=7315f7c7db09430a962b9b45c222cfd1)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:21.686313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecc79f-1680-4d28-8f0e-f800cedff6bc None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:21.688581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 176/364] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Aug 3 18:37:21 2022] GET /identity/v3/groups/7315f7c7db09430a962b9b45c222cfd1 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:21.692781 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecc79f-1680-4d28-8f0e-f800cedff6bc None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:21.701367 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b49c74-2625-48a2-913b-4bb3e9f907e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:21.701973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b49c74-2625-48a2-913b-4bb3e9f907e4 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:21.702211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b49c74-2625-48a2-913b-4bb3e9f907e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:21.702514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b49c74-2625-48a2-913b-4bb3e9f907e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:21.703704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b49c74-2625-48a2-913b-4bb3e9f907e4 None admin] RBAC: Adding query filter params (role.id=5f5e6a876c4a4fb0869e5bea07e7a65c, user.id=b008c7a092f14685ae026c8495b7966c, scope.system=all) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:21.704021 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b49c74-2625-48a2-913b-4bb3e9f907e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:21.705893 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b49c74-2625-48a2-913b-4bb3e9f907e4 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:21.723168 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 189/365] 10.222.0.69 () {56 vars in 1308 bytes} [Wed Aug 3 18:37:21 2022] GET /identity/v3/role_assignments?user.id=b008c7a092f14685ae026c8495b7966c&scope.system=all&role.id=5f5e6a876c4a4fb0869e5bea07e7a65c => generated 510 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:21.724510 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351f456-149e-4439-8a0b-8e0c00d3176b None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:21.724880 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351f456-149e-4439-8a0b-8e0c00d3176b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:21.724984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351f456-149e-4439-8a0b-8e0c00d3176b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:21.746067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-8351f456-149e-4439-8a0b-8e0c00d3176b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:21.747424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8351f456-149e-4439-8a0b-8e0c00d3176b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:21.772783 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:21.845797 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 190/366] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:21.846803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351f456-149e-4439-8a0b-8e0c00d3176b None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:21.860993 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:21.874619 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecc79f-1680-4d28-8f0e-f800cedff6bc None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:21.928573 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f59502-65fc-4ae4-bd76-ef2c340fda50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:21.929449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59502-65fc-4ae4-bd76-ef2c340fda50 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:21.929853 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59502-65fc-4ae4-bd76-ef2c340fda50 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:21.930247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59502-65fc-4ae4-bd76-ef2c340fda50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:21.931247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f59502-65fc-4ae4-bd76-ef2c340fda50 None admin] RBAC: Adding query filter params (group.id=7315f7c7db09430a962b9b45c222cfd1, role.id=548e622a9ae745d99d4cd34e68c1652c, scope.project.id=42291678aa1842ff9db31b9d1e39dc2c) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:21.932307 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f59502-65fc-4ae4-bd76-ef2c340fda50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:21.934465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f59502-65fc-4ae4-bd76-ef2c340fda50 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:21.942474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 177/367] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Aug 3 18:37:21 2022] GET /identity/v3/role_assignments?group.id=7315f7c7db09430a962b9b45c222cfd1&scope.project.id=42291678aa1842ff9db31b9d1e39dc2c&role.id=548e622a9ae745d99d4cd34e68c1652c => generated 260 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:21.943204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f59502-65fc-4ae4-bd76-ef2c340fda50 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:22.022878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351f456-149e-4439-8a0b-8e0c00d3176b None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:22.080785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f489dc-be54-4a9a-a46d-4836f3d03c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:22.081459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f489dc-be54-4a9a-a46d-4836f3d03c83 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:22.081792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f489dc-be54-4a9a-a46d-4836f3d03c83 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:22.082080 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f489dc-be54-4a9a-a46d-4836f3d03c83 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:22.097774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f489dc-be54-4a9a-a46d-4836f3d03c83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:22.097966 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f59502-65fc-4ae4-bd76-ef2c340fda50 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-31f489dc-be54-4a9a-a46d-4836f3d03c83 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:22.104902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:22.105876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:22.107126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:22.107126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:22.107126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:22.107126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:22.107126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:22.107126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:22.107126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:22.107126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:22.107126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:22.107126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:22.107126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:22.107126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:22.107126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 191/368] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:37:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:22.109043 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f489dc-be54-4a9a-a46d-4836f3d03c83 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:22.126274 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:22.147771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0856ef-6629-4024-80df-fc37c027f256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:22.148492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0856ef-6629-4024-80df-fc37c027f256 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:22.148777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0856ef-6629-4024-80df-fc37c027f256 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:22.149047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0856ef-6629-4024-80df-fc37c027f256 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:22.150067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0856ef-6629-4024-80df-fc37c027f256 None admin] RBAC: Adding query filter params (group.id=7315f7c7db09430a962b9b45c222cfd1, role.id=f3b7ffe4d1f04af18f7da8d9a51031b3, scope.project.id=15984787df4448da8a1beaa8b7b15320) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:22.150625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0856ef-6629-4024-80df-fc37c027f256 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:22.153026 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0856ef-6629-4024-80df-fc37c027f256 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:22.161251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 178/369] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Aug 3 18:37:21 2022] GET /identity/v3/role_assignments?group.id=7315f7c7db09430a962b9b45c222cfd1&scope.project.id=15984787df4448da8a1beaa8b7b15320&role.id=f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:22.162253 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0856ef-6629-4024-80df-fc37c027f256 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:22.201312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:22.246659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f489dc-be54-4a9a-a46d-4836f3d03c83 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:22.314805 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494dc80a-021a-4a72-88e0-80ae80a235fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:22.317883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dc80a-021a-4a72-88e0-80ae80a235fb None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:22.318407 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dc80a-021a-4a72-88e0-80ae80a235fb None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:22.320601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dc80a-021a-4a72-88e0-80ae80a235fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:22.338902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494dc80a-021a-4a72-88e0-80ae80a235fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-494dc80a-021a-4a72-88e0-80ae80a235fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:22.345642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:22.346614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:22.347482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:22.347482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:22.347482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:22.347482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:22.347482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:22.347482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:22.347482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:22.347482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:22.347482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:22.347482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:22.347482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:37:22.347482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:22.353922 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 192/370] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:37:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:22.359831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494dc80a-021a-4a72-88e0-80ae80a235fb None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:22.367996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0856ef-6629-4024-80df-fc37c027f256 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:22.412739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309eb8f-02d9-4fb3-aca1-3eb8477fd0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:22.413763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309eb8f-02d9-4fb3-aca1-3eb8477fd0ce None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:22.414248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309eb8f-02d9-4fb3-aca1-3eb8477fd0ce None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:22.414719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309eb8f-02d9-4fb3-aca1-3eb8477fd0ce None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:22.415772 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309eb8f-02d9-4fb3-aca1-3eb8477fd0ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:22.416282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309eb8f-02d9-4fb3-aca1-3eb8477fd0ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:22.418102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309eb8f-02d9-4fb3-aca1-3eb8477fd0ce None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:22.427340 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 179/371] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:22.434427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309eb8f-02d9-4fb3-aca1-3eb8477fd0ce None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:22.463070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:22.546859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:22.559903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494dc80a-021a-4a72-88e0-80ae80a235fb None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:22.573042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309eb8f-02d9-4fb3-aca1-3eb8477fd0ce None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:22.613785 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eccc70-b86a-416b-b424-1be192435211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:22.614774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eccc70-b86a-416b-b424-1be192435211 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:22.615259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eccc70-b86a-416b-b424-1be192435211 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:22.615733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eccc70-b86a-416b-b424-1be192435211 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:22.646504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212c89b-c898-484e-ab67-1f4ce3c7760f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:22.647130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212c89b-c898-484e-ab67-1f4ce3c7760f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:22.647421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212c89b-c898-484e-ab67-1f4ce3c7760f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:22.647722 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212c89b-c898-484e-ab67-1f4ce3c7760f None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:22.648489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212c89b-c898-484e-ab67-1f4ce3c7760f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:22.648792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212c89b-c898-484e-ab67-1f4ce3c7760f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:22.650230 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212c89b-c898-484e-ab67-1f4ce3c7760f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:22.659953 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 193/372] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:22.665407 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212c89b-c898-484e-ab67-1f4ce3c7760f None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:22.708517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eccc70-b86a-416b-b424-1be192435211 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:22.711075 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eccc70-b86a-416b-b424-1be192435211 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:22.805880 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212c89b-c898-484e-ab67-1f4ce3c7760f None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:22.814711 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:22.841681 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 180/373] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:22.845399 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:22.849187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eccc70-b86a-416b-b424-1be192435211 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:22.855171 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b9774-93cf-4dd2-9638-a7a8bc3ccb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:22.856447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b9774-93cf-4dd2-9638-a7a8bc3ccb0d None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:22.856943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b9774-93cf-4dd2-9638-a7a8bc3ccb0d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:22.857409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b9774-93cf-4dd2-9638-a7a8bc3ccb0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:22.988147 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b9774-93cf-4dd2-9638-a7a8bc3ccb0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:22.990257 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b9774-93cf-4dd2-9638-a7a8bc3ccb0d None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:23.045327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eccc70-b86a-416b-b424-1be192435211 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:23.092028 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abd93a-7e18-48f0-b133-dbcf69dd065f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:23.093039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abd93a-7e18-48f0-b133-dbcf69dd065f None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:23.093822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abd93a-7e18-48f0-b133-dbcf69dd065f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:23.094321 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abd93a-7e18-48f0-b133-dbcf69dd065f None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:23.095921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39abd93a-7e18-48f0-b133-dbcf69dd065f None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:23.098160 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39abd93a-7e18-48f0-b133-dbcf69dd065f None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:23.102947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 181/374] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:22 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:23.109912 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abd93a-7e18-48f0-b133-dbcf69dd065f None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:23.173733 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 194/375] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 18:37:23.176822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b9774-93cf-4dd2-9638-a7a8bc3ccb0d None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:23.190741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:23.233803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:23.282146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abd93a-7e18-48f0-b133-dbcf69dd065f None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:23.323523 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b9774-93cf-4dd2-9638-a7a8bc3ccb0d None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:23.364531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f882da7-c875-41c8-9706-bed32829b35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:23.364731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e3a43-93f5-44f8-b5b3-f81846a6a424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:23.365104 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f882da7-c875-41c8-9706-bed32829b35c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:23.365387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f882da7-c875-41c8-9706-bed32829b35c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:23.365665 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f882da7-c875-41c8-9706-bed32829b35c None admin] PATH_INFO: `/v3/groups/7315f7c7db09430a962b9b45c222cfd1` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:23.366343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e3a43-93f5-44f8-b5b3-f81846a6a424 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:23.366821 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e3a43-93f5-44f8-b5b3-f81846a6a424 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:23.367311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e3a43-93f5-44f8-b5b3-f81846a6a424 None admin] PATH_INFO: `/v3/projects/649429edefe44474b26fed9c93d5ec67` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:23.368777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e3a43-93f5-44f8-b5b3-f81846a6a424 None admin] RBAC: Authorizing `identity:get_project(project_id=649429edefe44474b26fed9c93d5ec67)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:23.368883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f882da7-c875-41c8-9706-bed32829b35c None admin] RBAC: Authorizing `identity:get_group(group_id=7315f7c7db09430a962b9b45c222cfd1)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:23.370362 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f882da7-c875-41c8-9706-bed32829b35c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:23.371060 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e3a43-93f5-44f8-b5b3-f81846a6a424 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:23.373585 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 195/377] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:23 2022] GET /identity/v3/projects/649429edefe44474b26fed9c93d5ec67 => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:23.377101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 182/377] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Aug 3 18:37:23 2022] GET /identity/v3/groups/7315f7c7db09430a962b9b45c222cfd1 => generated 226 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:23.378230 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e3a43-93f5-44f8-b5b3-f81846a6a424 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:23.392669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f882da7-c875-41c8-9706-bed32829b35c None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:23.495811 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:23.501125 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e3a43-93f5-44f8-b5b3-f81846a6a424 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:23.539035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f21580-8a50-4ead-82a0-c1b5c2eb7aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:23.540119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f21580-8a50-4ead-82a0-c1b5c2eb7aaf None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:23.541344 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f21580-8a50-4ead-82a0-c1b5c2eb7aaf None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:23.541852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f21580-8a50-4ead-82a0-c1b5c2eb7aaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:23.542932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f21580-8a50-4ead-82a0-c1b5c2eb7aaf None admin] RBAC: Adding query filter params (group.id=7315f7c7db09430a962b9b45c222cfd1, role.id=548e622a9ae745d99d4cd34e68c1652c, scope.project.id=15984787df4448da8a1beaa8b7b15320) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:23.543506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f21580-8a50-4ead-82a0-c1b5c2eb7aaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:23.545330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f21580-8a50-4ead-82a0-c1b5c2eb7aaf None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:23.551669 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 196/378] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Aug 3 18:37:23 2022] GET /identity/v3/role_assignments?group.id=7315f7c7db09430a962b9b45c222cfd1&scope.project.id=15984787df4448da8a1beaa8b7b15320&role.id=548e622a9ae745d99d4cd34e68c1652c => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:23.592035 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f882da7-c875-41c8-9706-bed32829b35c None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:23.611025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:23.635435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5151bd1-5173-488b-828f-2b6b40d49d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:23.636165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5151bd1-5173-488b-828f-2b6b40d49d14 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:23.636503 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5151bd1-5173-488b-828f-2b6b40d49d14 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:23.636782 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5151bd1-5173-488b-828f-2b6b40d49d14 None admin] PATH_INFO: `/v3/groups/d01c88b6d1ed4973ac411d195d7c9e1b` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:23.643987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5151bd1-5173-488b-828f-2b6b40d49d14 None admin] RBAC: Authorizing `identity:get_group(group_id=d01c88b6d1ed4973ac411d195d7c9e1b)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:23.645506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5151bd1-5173-488b-828f-2b6b40d49d14 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:23.647641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 183/379] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Aug 3 18:37:23 2022] GET /identity/v3/groups/d01c88b6d1ed4973ac411d195d7c9e1b => generated 229 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:23.654182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f21580-8a50-4ead-82a0-c1b5c2eb7aaf None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:23.800101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f21580-8a50-4ead-82a0-c1b5c2eb7aaf None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:23.842214 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6c9a5-ec4a-4c04-8563-c1ffc6a1deeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:23.843567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6c9a5-ec4a-4c04-8563-c1ffc6a1deeb None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:23.844000 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6c9a5-ec4a-4c04-8563-c1ffc6a1deeb None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:23.844468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6c9a5-ec4a-4c04-8563-c1ffc6a1deeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:23.845460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6c9a5-ec4a-4c04-8563-c1ffc6a1deeb None admin] RBAC: Adding query filter params (group.id=d01c88b6d1ed4973ac411d195d7c9e1b, role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=649429edefe44474b26fed9c93d5ec67) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:23.845928 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6c9a5-ec4a-4c04-8563-c1ffc6a1deeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:23.847611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6c9a5-ec4a-4c04-8563-c1ffc6a1deeb None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:23.855484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 197/380] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Aug 3 18:37:23 2022] GET /identity/v3/role_assignments?group.id=d01c88b6d1ed4973ac411d195d7c9e1b&scope.project.id=649429edefe44474b26fed9c93d5ec67&role.id=19557e941d6f475ba92cf862846f9872 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:23.972258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3cd97-4ed2-40e8-b3d2-6e3413e55384 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:23.972604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3cd97-4ed2-40e8-b3d2-6e3413e55384 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:23.972852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3cd97-4ed2-40e8-b3d2-6e3413e55384 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:23.974542 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 184/381] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:23.980267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0133699-81b2-45cb-9124-53e4941e91f9 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:23.980907 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0133699-81b2-45cb-9124-53e4941e91f9 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:23.981395 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0133699-81b2-45cb-9124-53e4941e91f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:24.004882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-e0133699-81b2-45cb-9124-53e4941e91f9 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:24.006476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e0133699-81b2-45cb-9124-53e4941e91f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:24.038832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:24.087817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 198/382] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:24.115435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3cd97-4ed2-40e8-b3d2-6e3413e55384 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:24.265198 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3cd97-4ed2-40e8-b3d2-6e3413e55384 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:24.317470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4d000-3e65-4060-8975-79b85f54dc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:24.318124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4d000-3e65-4060-8975-79b85f54dc2a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:24.318407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4d000-3e65-4060-8975-79b85f54dc2a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:24.318682 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4d000-3e65-4060-8975-79b85f54dc2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:24.331700 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4d000-3e65-4060-8975-79b85f54dc2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-dad4d000-3e65-4060-8975-79b85f54dc2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:24.336680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:24.337659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:24.338536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:24.338536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:24.338536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:24.338536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:24.338536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:24.338536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:24.338536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:24.338536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:24.338536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:24.338536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:24.338536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:24.338536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:24.342981 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 185/383] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:24.345398 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0133699-81b2-45cb-9124-53e4941e91f9 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:24.492189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0133699-81b2-45cb-9124-53e4941e91f9 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:24.530078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ea89d-2a12-4b4c-9721-477812be2d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:24.531060 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ea89d-2a12-4b4c-9721-477812be2d50 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:24.531568 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ea89d-2a12-4b4c-9721-477812be2d50 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:24.532075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ea89d-2a12-4b4c-9721-477812be2d50 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:24.533057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ea89d-2a12-4b4c-9721-477812be2d50 None admin] RBAC: Adding query filter params (name=member) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:24.533579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ea89d-2a12-4b4c-9721-477812be2d50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:24.535160 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ea89d-2a12-4b4c-9721-477812be2d50 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:24.544821 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 199/384] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:24.553369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4d000-3e65-4060-8975-79b85f54dc2a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:24.589438 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a28ae7-0fe3-46a9-9ae3-35918d35c468 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:24.589757 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a28ae7-0fe3-46a9-9ae3-35918d35c468 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:24.590001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a28ae7-0fe3-46a9-9ae3-35918d35c468 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:24.591507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 200/385] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:37:24.597221 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b3480-9367-4ba0-8507-72a9b335e70d None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:24.597505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b3480-9367-4ba0-8507-72a9b335e70d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:24.597784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b3480-9367-4ba0-8507-72a9b335e70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:24.632360 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d28b3480-9367-4ba0-8507-72a9b335e70d None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:24.634917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d28b3480-9367-4ba0-8507-72a9b335e70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:24.656521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:24.666879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:24.760995 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4d000-3e65-4060-8975-79b85f54dc2a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:24.779210 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 201/386] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:24.810069 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b3480-9367-4ba0-8507-72a9b335e70d None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:24.819441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acf2b6-2a0e-48a3-99f2-72c682614865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:24.822746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acf2b6-2a0e-48a3-99f2-72c682614865 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:24.824588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acf2b6-2a0e-48a3-99f2-72c682614865 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:24.825738 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acf2b6-2a0e-48a3-99f2-72c682614865 None admin] PATH_INFO: `/v3/groups/7315f7c7db09430a962b9b45c222cfd1` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:24.828700 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acf2b6-2a0e-48a3-99f2-72c682614865 None admin] RBAC: Authorizing `identity:get_group(group_id=7315f7c7db09430a962b9b45c222cfd1)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:24.831121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acf2b6-2a0e-48a3-99f2-72c682614865 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:24.833747 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 186/387] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Aug 3 18:37:24 2022] GET /identity/v3/groups/7315f7c7db09430a962b9b45c222cfd1 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:24.840396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acf2b6-2a0e-48a3-99f2-72c682614865 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:24.977044 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b3480-9367-4ba0-8507-72a9b335e70d None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:25.001905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acf2b6-2a0e-48a3-99f2-72c682614865 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:25.019459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:25.024553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6f279-a905-47e0-be4e-9151696dddfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:25.025868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6f279-a905-47e0-be4e-9151696dddfd None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:25.026153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6f279-a905-47e0-be4e-9151696dddfd None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:25.026426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6f279-a905-47e0-be4e-9151696dddfd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:25.039075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6f279-a905-47e0-be4e-9151696dddfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-e8e6f279-a905-47e0-be4e-9151696dddfd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:25.046458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:25.047363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:25.048866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:25.048866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:25.048866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:25.048866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:25.048866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:25.048866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:25.048866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:25.048866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:25.048866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:25.048866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:25.048866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:25.048866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:25.048866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 202/388] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:37:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:25.049575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bf222-d680-4714-93ac-d051cddba8bb None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:25.049575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bf222-d680-4714-93ac-d051cddba8bb None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:25.049703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bf222-d680-4714-93ac-d051cddba8bb None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:25.051133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 203/389] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:25.056673 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bf222-d680-4714-93ac-d051cddba8bb None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:25.061931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafedf5-1bad-4fed-b040-1f5255c5369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:25.062901 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafedf5-1bad-4fed-b040-1f5255c5369a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:25.063379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafedf5-1bad-4fed-b040-1f5255c5369a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:25.063855 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafedf5-1bad-4fed-b040-1f5255c5369a None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:25.065300 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafedf5-1bad-4fed-b040-1f5255c5369a None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:25.066958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafedf5-1bad-4fed-b040-1f5255c5369a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:25.072654 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:25.073740 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 187/390] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:24 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:25.075700 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca94b4-b39d-4863-ab9f-5980c3860c55 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:25.076986 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca94b4-b39d-4863-ab9f-5980c3860c55 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:25.077482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca94b4-b39d-4863-ab9f-5980c3860c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:25.104923 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-28ca94b4-b39d-4863-ab9f-5980c3860c55 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:25.106826 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-28ca94b4-b39d-4863-ab9f-5980c3860c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:25.196194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 188/391] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:25.197833 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca94b4-b39d-4863-ab9f-5980c3860c55 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:25.200475 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bf222-d680-4714-93ac-d051cddba8bb None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:25.244422 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a7732-fbdb-408f-96c6-29bf5e75c23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:25.245874 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a7732-fbdb-408f-96c6-29bf5e75c23c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:25.246380 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a7732-fbdb-408f-96c6-29bf5e75c23c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:25.247253 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a7732-fbdb-408f-96c6-29bf5e75c23c None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:25.249667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a7732-fbdb-408f-96c6-29bf5e75c23c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:25.250258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a7732-fbdb-408f-96c6-29bf5e75c23c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:25.253150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a7732-fbdb-408f-96c6-29bf5e75c23c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:25.265744 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 204/392] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:25.267614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a7732-fbdb-408f-96c6-29bf5e75c23c None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:25.341193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca94b4-b39d-4863-ab9f-5980c3860c55 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:25.357176 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:25.376081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:25.401118 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e46e5b-79e1-4f7d-ac70-193f333cc5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:25.402222 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e46e5b-79e1-4f7d-ac70-193f333cc5ce None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:25.402379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e46e5b-79e1-4f7d-ac70-193f333cc5ce None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:25.402610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e46e5b-79e1-4f7d-ac70-193f333cc5ce None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c/groups/7315f7c7db09430a962b9b45c222cfd1/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:25.404841 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e46e5b-79e1-4f7d-ac70-193f333cc5ce None admin] RBAC: Authorizing `identity:create_grant(project_id=42291678aa1842ff9db31b9d1e39dc2c, group_id=7315f7c7db09430a962b9b45c222cfd1, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:25.406660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e46e5b-79e1-4f7d-ac70-193f333cc5ce None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:25.419012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a7732-fbdb-408f-96c6-29bf5e75c23c None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:25.426696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 189/393] 10.222.0.69 () {58 vars in 1442 bytes} [Wed Aug 3 18:37:25 2022] PUT /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c/groups/7315f7c7db09430a962b9b45c222cfd1/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:37:25.433369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e46e5b-79e1-4f7d-ac70-193f333cc5ce None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:25.473202 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07be06-9512-40b6-9e4b-9cc57f791134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:25.474290 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07be06-9512-40b6-9e4b-9cc57f791134 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:25.474832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07be06-9512-40b6-9e4b-9cc57f791134 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:25.475306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07be06-9512-40b6-9e4b-9cc57f791134 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:25.489125 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07be06-9512-40b6-9e4b-9cc57f791134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-fc07be06-9512-40b6-9e4b-9cc57f791134 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:25.497687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:25.498643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:25.499471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:25.499471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:25.499471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:25.499471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:25.499471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:25.499471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:25.499471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:25.499471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:25.499471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:25.499471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:25.499471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:25.499471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:25.502539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 205/394] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:25.508168 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07be06-9512-40b6-9e4b-9cc57f791134 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:25.670501 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07be06-9512-40b6-9e4b-9cc57f791134 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:25.690534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e46e5b-79e1-4f7d-ac70-193f333cc5ce None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:25.703575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:25.711031 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b7afa-12ad-4b3e-9db2-1abaa80f445b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:25.711556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b7afa-12ad-4b3e-9db2-1abaa80f445b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:25.711842 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b7afa-12ad-4b3e-9db2-1abaa80f445b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:25.712122 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b7afa-12ad-4b3e-9db2-1abaa80f445b None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:25.713127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b7afa-12ad-4b3e-9db2-1abaa80f445b None admin] RBAC: Adding query filter params (name=member) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:25.713444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b7afa-12ad-4b3e-9db2-1abaa80f445b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:25.714825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b7afa-12ad-4b3e-9db2-1abaa80f445b None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:25.724570 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 206/395] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:25.730512 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b7afa-12ad-4b3e-9db2-1abaa80f445b None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:25.735388 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25889c2f-6207-4236-87aa-83c442301794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:25.736252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25889c2f-6207-4236-87aa-83c442301794 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:25.736739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25889c2f-6207-4236-87aa-83c442301794 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:25.737198 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25889c2f-6207-4236-87aa-83c442301794 None admin] PATH_INFO: `/v3/groups/7315f7c7db09430a962b9b45c222cfd1` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:25.738537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25889c2f-6207-4236-87aa-83c442301794 None admin] RBAC: Authorizing `identity:get_group(group_id=7315f7c7db09430a962b9b45c222cfd1)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:25.740150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25889c2f-6207-4236-87aa-83c442301794 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:25.742148 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 190/396] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Aug 3 18:37:25 2022] GET /identity/v3/groups/7315f7c7db09430a962b9b45c222cfd1 => generated 226 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:25.749091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25889c2f-6207-4236-87aa-83c442301794 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:25.796977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:25.877382 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25889c2f-6207-4236-87aa-83c442301794 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:25.906256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b7afa-12ad-4b3e-9db2-1abaa80f445b None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:25.917405 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c9d65-0315-4b8d-87f9-d500e72e9401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:25.918216 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c9d65-0315-4b8d-87f9-d500e72e9401 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:25.918493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c9d65-0315-4b8d-87f9-d500e72e9401 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:25.918764 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c9d65-0315-4b8d-87f9-d500e72e9401 None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:25.919976 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c9d65-0315-4b8d-87f9-d500e72e9401 None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:25.921498 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c9d65-0315-4b8d-87f9-d500e72e9401 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:25.926012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 191/397] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:25 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:25.929567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c9d65-0315-4b8d-87f9-d500e72e9401 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:25.990732 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578c520-bbc9-4a6f-b87c-15102a32be1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:25.991478 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578c520-bbc9-4a6f-b87c-15102a32be1f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:25.991759 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578c520-bbc9-4a6f-b87c-15102a32be1f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:25.992039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578c520-bbc9-4a6f-b87c-15102a32be1f None admin] PATH_INFO: `/v3/groups/7315f7c7db09430a962b9b45c222cfd1` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:25.993294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578c520-bbc9-4a6f-b87c-15102a32be1f None admin] RBAC: Authorizing `identity:get_group(group_id=7315f7c7db09430a962b9b45c222cfd1)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:25.994770 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578c520-bbc9-4a6f-b87c-15102a32be1f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:26.001163 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 207/398] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Aug 3 18:37:25 2022] GET /identity/v3/groups/7315f7c7db09430a962b9b45c222cfd1 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:26.003072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578c520-bbc9-4a6f-b87c-15102a32be1f None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:26.071391 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c9d65-0315-4b8d-87f9-d500e72e9401 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:26.083786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:26.110900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59acec-6438-4cb4-9d43-8879d293873c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:26.111576 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59acec-6438-4cb4-9d43-8879d293873c None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:26.111856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59acec-6438-4cb4-9d43-8879d293873c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:26.112128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59acec-6438-4cb4-9d43-8879d293873c None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c/groups/7315f7c7db09430a962b9b45c222cfd1/roles/548e622a9ae745d99d4cd34e68c1652c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:26.114583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59acec-6438-4cb4-9d43-8879d293873c None admin] RBAC: Authorizing `identity:create_grant(project_id=42291678aa1842ff9db31b9d1e39dc2c, group_id=7315f7c7db09430a962b9b45c222cfd1, role_id=548e622a9ae745d99d4cd34e68c1652c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:26.116019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59acec-6438-4cb4-9d43-8879d293873c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:26.132614 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 192/399] 10.222.0.69 () {58 vars in 1442 bytes} [Wed Aug 3 18:37:25 2022] PUT /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c/groups/7315f7c7db09430a962b9b45c222cfd1/roles/548e622a9ae745d99d4cd34e68c1652c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:37:26.232859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:26.298726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0edd4-2770-4792-abc0-ab6fbcb2123f None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:26.299279 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0edd4-2770-4792-abc0-ab6fbcb2123f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:26.299279 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0edd4-2770-4792-abc0-ab6fbcb2123f None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:26.300972 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 193/400] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:37:26.302779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578c520-bbc9-4a6f-b87c-15102a32be1f None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:26.306485 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09e5f3-5ad8-4c67-bc39-594c7d865740 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:26.306908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09e5f3-5ad8-4c67-bc39-594c7d865740 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:26.307148 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09e5f3-5ad8-4c67-bc39-594c7d865740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:26.340730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-fe09e5f3-5ad8-4c67-bc39-594c7d865740 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:26.341930 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-fe09e5f3-5ad8-4c67-bc39-594c7d865740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:26.356478 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba40064-eb46-4781-8071-52ece341e7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:26.357473 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba40064-eb46-4781-8071-52ece341e7fb None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:26.357917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba40064-eb46-4781-8071-52ece341e7fb None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:26.358339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba40064-eb46-4781-8071-52ece341e7fb None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:26.359877 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba40064-eb46-4781-8071-52ece341e7fb None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:26.361568 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba40064-eb46-4781-8071-52ece341e7fb None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:26.363579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 208/401] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:26 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:26.369478 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba40064-eb46-4781-8071-52ece341e7fb None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:26.441457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 194/402] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:26.442746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5b904-2d90-4861-9e46-6e45c2b33950 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:26.443041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5b904-2d90-4861-9e46-6e45c2b33950 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:26.443320 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5b904-2d90-4861-9e46-6e45c2b33950 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:26.444669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 195/403] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:26.450709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa76066-336d-4c14-bbf2-60c0ebdea93f None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:26.451001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa76066-336d-4c14-bbf2-60c0ebdea93f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:26.451277 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa76066-336d-4c14-bbf2-60c0ebdea93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:26.470378 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-3fa76066-336d-4c14-bbf2-60c0ebdea93f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:26.471570 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-3fa76066-336d-4c14-bbf2-60c0ebdea93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:26.519109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba40064-eb46-4781-8071-52ece341e7fb None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:26.553339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:26.554877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 196/404] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:26.560343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ae2ee-71d5-4dad-9cda-d00a846296a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:26.560343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ae2ee-71d5-4dad-9cda-d00a846296a4 None admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:26.560594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ae2ee-71d5-4dad-9cda-d00a846296a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:26.560793 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ae2ee-71d5-4dad-9cda-d00a846296a4 None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320/groups/7315f7c7db09430a962b9b45c222cfd1/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:26.562248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa76066-336d-4c14-bbf2-60c0ebdea93f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:26.566979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ae2ee-71d5-4dad-9cda-d00a846296a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=15984787df4448da8a1beaa8b7b15320, group_id=7315f7c7db09430a962b9b45c222cfd1, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:26.566979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ae2ee-71d5-4dad-9cda-d00a846296a4 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:26.582033 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 209/405] 10.222.0.69 () {58 vars in 1442 bytes} [Wed Aug 3 18:37:26 2022] PUT /identity/v3/projects/15984787df4448da8a1beaa8b7b15320/groups/7315f7c7db09430a962b9b45c222cfd1/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:37:26.596514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ae2ee-71d5-4dad-9cda-d00a846296a4 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:26.601971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:26.689492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa76066-336d-4c14-bbf2-60c0ebdea93f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:26.721739 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ae2ee-71d5-4dad-9cda-d00a846296a4 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:26.727663 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45b15d-9498-4934-b3e3-57c24b978990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:26.729269 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45b15d-9498-4934-b3e3-57c24b978990 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:26.729715 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45b15d-9498-4934-b3e3-57c24b978990 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:26.730152 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45b15d-9498-4934-b3e3-57c24b978990 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:26.742806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45b15d-9498-4934-b3e3-57c24b978990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-bd45b15d-9498-4934-b3e3-57c24b978990 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:26.749210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:26.750287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:26.751174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:26.751174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:26.751174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:26.751174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:26.751174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:26.751174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:26.751174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:26.751174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:26.751174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:26.751174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:26.751174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:26.751174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:26.753972 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 197/406] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:37:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:26.765205 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05313d-97f5-42cc-980e-eb2355bfdcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:26.765753 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45b15d-9498-4934-b3e3-57c24b978990 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:26.765928 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05313d-97f5-42cc-980e-eb2355bfdcc7 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:26.766216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05313d-97f5-42cc-980e-eb2355bfdcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:26.766550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05313d-97f5-42cc-980e-eb2355bfdcc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:26.784406 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05313d-97f5-42cc-980e-eb2355bfdcc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-0e05313d-97f5-42cc-980e-eb2355bfdcc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:26.787082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:26.788178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:26.789222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:26.789222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:26.789222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:26.789222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:26.789222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:26.789222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:26.789222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:26.789222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:26.789222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:26.789222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:26.789222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:37:26.789222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:26.789222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 210/407] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:37:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:26.795245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05313d-97f5-42cc-980e-eb2355bfdcc7 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:26.853786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:26.908897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05313d-97f5-42cc-980e-eb2355bfdcc7 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:26.910045 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:26.940499 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45b15d-9498-4934-b3e3-57c24b978990 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:26.975256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a8609-dcf7-4d5e-809d-3870ae38e882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:26.975900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a8609-dcf7-4d5e-809d-3870ae38e882 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:26.976203 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a8609-dcf7-4d5e-809d-3870ae38e882 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:26.976477 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a8609-dcf7-4d5e-809d-3870ae38e882 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:26.977223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a8609-dcf7-4d5e-809d-3870ae38e882 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:26.977525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a8609-dcf7-4d5e-809d-3870ae38e882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:26.979111 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a8609-dcf7-4d5e-809d-3870ae38e882 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:26.986864 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61c7e1-83f5-4015-acdf-517e7f803271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:26.988534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61c7e1-83f5-4015-acdf-517e7f803271 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:26.988815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61c7e1-83f5-4015-acdf-517e7f803271 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:26.989087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61c7e1-83f5-4015-acdf-517e7f803271 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:26.989821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca61c7e1-83f5-4015-acdf-517e7f803271 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:26.990122 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca61c7e1-83f5-4015-acdf-517e7f803271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:26.996433 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca61c7e1-83f5-4015-acdf-517e7f803271 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:27.002750 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 211/408] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:27.008549 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a8609-dcf7-4d5e-809d-3870ae38e882 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:27.011190 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 198/409] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:27.020226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61c7e1-83f5-4015-acdf-517e7f803271 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:27.211999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61c7e1-83f5-4015-acdf-517e7f803271 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:27.216046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a8609-dcf7-4d5e-809d-3870ae38e882 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:27.242359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:27.257349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da61f03-69fd-4bbf-b395-f5dbd68ac8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:27.258004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da61f03-69fd-4bbf-b395-f5dbd68ac8af None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:27.258284 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da61f03-69fd-4bbf-b395-f5dbd68ac8af None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:27.258560 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da61f03-69fd-4bbf-b395-f5dbd68ac8af None admin] PATH_INFO: `/v3/groups/7315f7c7db09430a962b9b45c222cfd1` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:27.259881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da61f03-69fd-4bbf-b395-f5dbd68ac8af None admin] RBAC: Authorizing `identity:get_group(group_id=7315f7c7db09430a962b9b45c222cfd1)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:27.261600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da61f03-69fd-4bbf-b395-f5dbd68ac8af None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:27.264126 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 199/410] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Aug 3 18:37:27 2022] GET /identity/v3/groups/7315f7c7db09430a962b9b45c222cfd1 => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:27.270128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da61f03-69fd-4bbf-b395-f5dbd68ac8af None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:27.295629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:27.295998 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d5f41-6957-4dd1-8ab9-aae2f18ad15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:27.297840 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d5f41-6957-4dd1-8ab9-aae2f18ad15c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:27.299579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d5f41-6957-4dd1-8ab9-aae2f18ad15c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:27.299877 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d5f41-6957-4dd1-8ab9-aae2f18ad15c None admin] PATH_INFO: `/v3/groups/d01c88b6d1ed4973ac411d195d7c9e1b` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:27.301783 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d5f41-6957-4dd1-8ab9-aae2f18ad15c None admin] RBAC: Authorizing `identity:get_group(group_id=d01c88b6d1ed4973ac411d195d7c9e1b)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:27.303230 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d5f41-6957-4dd1-8ab9-aae2f18ad15c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:27.304862 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 212/411] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Aug 3 18:37:27 2022] GET /identity/v3/groups/d01c88b6d1ed4973ac411d195d7c9e1b => generated 229 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:27.313221 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d5f41-6957-4dd1-8ab9-aae2f18ad15c None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:27.419607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da61f03-69fd-4bbf-b395-f5dbd68ac8af None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:27.460262 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d598130-932a-486b-965e-92290a8a23be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:27.460932 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d598130-932a-486b-965e-92290a8a23be None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:27.461209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d598130-932a-486b-965e-92290a8a23be None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:27.461477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d598130-932a-486b-965e-92290a8a23be None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:27.467329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d598130-932a-486b-965e-92290a8a23be None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:27.468741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d598130-932a-486b-965e-92290a8a23be None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:27.470473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 200/412] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:27 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:27.476206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d598130-932a-486b-965e-92290a8a23be None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:27.507509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d5f41-6957-4dd1-8ab9-aae2f18ad15c None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:27.576981 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a387ada-5642-4d8f-a344-a2ce49628ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:27.577637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a387ada-5642-4d8f-a344-a2ce49628ce0 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:27.577922 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a387ada-5642-4d8f-a344-a2ce49628ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:27.578197 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a387ada-5642-4d8f-a344-a2ce49628ce0 None admin] PATH_INFO: `/v3/projects/649429edefe44474b26fed9c93d5ec67` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:27.582997 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a387ada-5642-4d8f-a344-a2ce49628ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=649429edefe44474b26fed9c93d5ec67)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:27.584504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a387ada-5642-4d8f-a344-a2ce49628ce0 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:27.586552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 213/413] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:27 2022] GET /identity/v3/projects/649429edefe44474b26fed9c93d5ec67 => generated 344 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:27.592744 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a387ada-5642-4d8f-a344-a2ce49628ce0 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:27.631389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:27.664553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:27.666541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d598130-932a-486b-965e-92290a8a23be None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:27.737735 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7250714-972c-4958-9c8f-bfd0174e2298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:27.738336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7250714-972c-4958-9c8f-bfd0174e2298 None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:27.738617 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7250714-972c-4958-9c8f-bfd0174e2298 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:27.738891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7250714-972c-4958-9c8f-bfd0174e2298 None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320/groups/7315f7c7db09430a962b9b45c222cfd1/roles/548e622a9ae745d99d4cd34e68c1652c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:27.742841 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7250714-972c-4958-9c8f-bfd0174e2298 None admin] RBAC: Authorizing `identity:create_grant(project_id=15984787df4448da8a1beaa8b7b15320, group_id=7315f7c7db09430a962b9b45c222cfd1, role_id=548e622a9ae745d99d4cd34e68c1652c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:27.744423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7250714-972c-4958-9c8f-bfd0174e2298 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:27.756413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a387ada-5642-4d8f-a344-a2ce49628ce0 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:27.765806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 201/414] 10.222.0.69 () {58 vars in 1442 bytes} [Wed Aug 3 18:37:27 2022] PUT /identity/v3/projects/15984787df4448da8a1beaa8b7b15320/groups/7315f7c7db09430a962b9b45c222cfd1/roles/548e622a9ae745d99d4cd34e68c1652c => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:37:27.816382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca1a42-088d-4002-bbef-62b1825b5587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:27.817093 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca1a42-088d-4002-bbef-62b1825b5587 None admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:27.817375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca1a42-088d-4002-bbef-62b1825b5587 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:27.817650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca1a42-088d-4002-bbef-62b1825b5587 None admin] PATH_INFO: `/v3/projects/649429edefe44474b26fed9c93d5ec67/groups/d01c88b6d1ed4973ac411d195d7c9e1b/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:27.819662 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca1a42-088d-4002-bbef-62b1825b5587 None admin] RBAC: Authorizing `identity:create_grant(project_id=649429edefe44474b26fed9c93d5ec67, group_id=d01c88b6d1ed4973ac411d195d7c9e1b, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:27.821149 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca1a42-088d-4002-bbef-62b1825b5587 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:27.841430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 214/415] 10.222.0.69 () {58 vars in 1442 bytes} [Wed Aug 3 18:37:27 2022] PUT /identity/v3/projects/649429edefe44474b26fed9c93d5ec67/groups/d01c88b6d1ed4973ac411d195d7c9e1b/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:37:28.061604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250bbe3-52b5-4eb6-8f1d-773c9c997894 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:28.061893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250bbe3-52b5-4eb6-8f1d-773c9c997894 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:28.062152 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250bbe3-52b5-4eb6-8f1d-773c9c997894 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:28.063775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 202/416] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:28.070292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41661252-d657-43e4-8754-ef130298a1d0 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:28.071397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41661252-d657-43e4-8754-ef130298a1d0 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:28.071397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41661252-d657-43e4-8754-ef130298a1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:28.097327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-41661252-d657-43e4-8754-ef130298a1d0 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:28.099062 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-41661252-d657-43e4-8754-ef130298a1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:28.175978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 215/417] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:28.216620 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250bbe3-52b5-4eb6-8f1d-773c9c997894 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:28.337268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250bbe3-52b5-4eb6-8f1d-773c9c997894 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:28.373698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73330c-7596-427b-9047-635be4546f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:28.374351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73330c-7596-427b-9047-635be4546f3e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:28.374629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73330c-7596-427b-9047-635be4546f3e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:28.374902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73330c-7596-427b-9047-635be4546f3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:28.387691 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73330c-7596-427b-9047-635be4546f3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-dd73330c-7596-427b-9047-635be4546f3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:28.392675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:28.394822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:28.396893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:28.396893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:28.396893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:28.396893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:28.396893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:28.396893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:28.396893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:28.396893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:28.396893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:28.396893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:28.396893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:28.396893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:28.400425 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 203/418] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:28.404336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41661252-d657-43e4-8754-ef130298a1d0 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:28.450368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:28.537924 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41661252-d657-43e4-8754-ef130298a1d0 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:28.575414 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fecbd-f7da-46dd-8545-83e50bc2a15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:28.576043 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fecbd-f7da-46dd-8545-83e50bc2a15a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:28.576359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fecbd-f7da-46dd-8545-83e50bc2a15a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:28.576637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fecbd-f7da-46dd-8545-83e50bc2a15a None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:28.577365 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fecbd-f7da-46dd-8545-83e50bc2a15a None admin] RBAC: Adding query filter params (name=member) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:28.577669 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fecbd-f7da-46dd-8545-83e50bc2a15a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:28.579041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fecbd-f7da-46dd-8545-83e50bc2a15a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:28.588606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 216/419] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:28.594209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73330c-7596-427b-9047-635be4546f3e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:28.597674 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:28.736602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73330c-7596-427b-9047-635be4546f3e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:28.794112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47009f46-13c2-4bbc-9f54-97abfbdbc13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:28.794829 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47009f46-13c2-4bbc-9f54-97abfbdbc13a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:28.795112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47009f46-13c2-4bbc-9f54-97abfbdbc13a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:28.795385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47009f46-13c2-4bbc-9f54-97abfbdbc13a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:28.804129 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3a758-b090-48ac-b930-67ca23a2f0ab None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:28.804444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3a758-b090-48ac-b930-67ca23a2f0ab None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:28.804723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3a758-b090-48ac-b930-67ca23a2f0ab None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:28.808603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 217/420] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:28.812929 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084bf94-70d9-4f31-a1ae-c1a56ea3b937 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:28.813212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084bf94-70d9-4f31-a1ae-c1a56ea3b937 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:28.813487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084bf94-70d9-4f31-a1ae-c1a56ea3b937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:28.832258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-2084bf94-70d9-4f31-a1ae-c1a56ea3b937 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:28.833453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2084bf94-70d9-4f31-a1ae-c1a56ea3b937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:28.883761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47009f46-13c2-4bbc-9f54-97abfbdbc13a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:28.886094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47009f46-13c2-4bbc-9f54-97abfbdbc13a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:28.887942 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:29.006852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 218/421] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:29.011462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 204/422] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:29.017327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084bf94-70d9-4f31-a1ae-c1a56ea3b937 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:29.032199 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47009f46-13c2-4bbc-9f54-97abfbdbc13a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:29.050509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:29.189696 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084bf94-70d9-4f31-a1ae-c1a56ea3b937 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:29.202744 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47009f46-13c2-4bbc-9f54-97abfbdbc13a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:29.204633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:29.241379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dadf26-67aa-4cf9-9994-1c290b309923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:29.242044 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dadf26-67aa-4cf9-9994-1c290b309923 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:29.242322 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dadf26-67aa-4cf9-9994-1c290b309923 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:29.242595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dadf26-67aa-4cf9-9994-1c290b309923 None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:29.246214 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dadf26-67aa-4cf9-9994-1c290b309923 None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:29.247656 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dadf26-67aa-4cf9-9994-1c290b309923 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:29.249368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 219/423] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:29 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:29.255436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dadf26-67aa-4cf9-9994-1c290b309923 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:29.276078 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0ecfe-1329-40fc-a2d5-d596910dcdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:29.277631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0ecfe-1329-40fc-a2d5-d596910dcdc0 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:29.277914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0ecfe-1329-40fc-a2d5-d596910dcdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:29.278191 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0ecfe-1329-40fc-a2d5-d596910dcdc0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:29.291137 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0ecfe-1329-40fc-a2d5-d596910dcdc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-68c0ecfe-1329-40fc-a2d5-d596910dcdc0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:29.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:29.298003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:29.298814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:29.298814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:29.298814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:29.298814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:29.298814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:29.298814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:29.298814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:29.298814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:29.298814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:29.298814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:29.298814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:29.298814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:29.303252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 205/424] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:37:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:29.304630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0ecfe-1329-40fc-a2d5-d596910dcdc0 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:29.447447 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0ecfe-1329-40fc-a2d5-d596910dcdc0 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:29.481083 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:29.487468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dadf26-67aa-4cf9-9994-1c290b309923 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:29.499479 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc247d7-b3f1-4475-aa72-a8344b0d8c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:29.500658 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc247d7-b3f1-4475-aa72-a8344b0d8c52 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:29.501135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc247d7-b3f1-4475-aa72-a8344b0d8c52 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:29.501885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc247d7-b3f1-4475-aa72-a8344b0d8c52 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:29.503360 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc247d7-b3f1-4475-aa72-a8344b0d8c52 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:29.504355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc247d7-b3f1-4475-aa72-a8344b0d8c52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:29.506524 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc247d7-b3f1-4475-aa72-a8344b0d8c52 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:29.519045 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 206/425] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:29.520623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bf89f-95b9-4091-a544-9bb0d616e535 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:29.521066 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bf89f-95b9-4091-a544-9bb0d616e535 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:29.521523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bf89f-95b9-4091-a544-9bb0d616e535 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:29.527941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 207/426] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:29.527941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03bf89f-95b9-4091-a544-9bb0d616e535 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:29.534608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5f0de-f0b3-481f-b83c-7528fe3df621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:29.535251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5f0de-f0b3-481f-b83c-7528fe3df621 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:29.535532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5f0de-f0b3-481f-b83c-7528fe3df621 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:29.535809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5f0de-f0b3-481f-b83c-7528fe3df621 None admin] PATH_INFO: `/v3/groups/7315f7c7db09430a962b9b45c222cfd1` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:29.537016 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5f0de-f0b3-481f-b83c-7528fe3df621 None admin] RBAC: Authorizing `identity:get_group(group_id=7315f7c7db09430a962b9b45c222cfd1)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:29.538451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5f0de-f0b3-481f-b83c-7528fe3df621 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:29.540449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 220/427] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Aug 3 18:37:29 2022] GET /identity/v3/groups/7315f7c7db09430a962b9b45c222cfd1 => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:29.541671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1dda6-de8d-42fd-b27e-bf48c8b94ac8 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:29.541959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1dda6-de8d-42fd-b27e-bf48c8b94ac8 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:29.542232 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1dda6-de8d-42fd-b27e-bf48c8b94ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:29.559793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:29.560326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-32d1dda6-de8d-42fd-b27e-bf48c8b94ac8 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:29.561524 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-32d1dda6-de8d-42fd-b27e-bf48c8b94ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:29.647372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 221/428] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:29.647372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1dda6-de8d-42fd-b27e-bf48c8b94ac8 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:29.662323 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03bf89f-95b9-4091-a544-9bb0d616e535 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:29.710913 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e0047-b6af-476f-af14-38750ed678fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:29.711914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e0047-b6af-476f-af14-38750ed678fa None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:29.713063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e0047-b6af-476f-af14-38750ed678fa None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:29.713641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e0047-b6af-476f-af14-38750ed678fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:29.754932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:29.763150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1dda6-de8d-42fd-b27e-bf48c8b94ac8 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:29.801549 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59588e2f-f083-4105-ad1f-d7c8348adff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:29.802172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59588e2f-f083-4105-ad1f-d7c8348adff9 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:29.802452 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59588e2f-f083-4105-ad1f-d7c8348adff9 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:29.802725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59588e2f-f083-4105-ad1f-d7c8348adff9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:29.803532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59588e2f-f083-4105-ad1f-d7c8348adff9 None admin] RBAC: Adding query filter params (group.id=7315f7c7db09430a962b9b45c222cfd1, role.id=f3b7ffe4d1f04af18f7da8d9a51031b3, scope.project.id=42291678aa1842ff9db31b9d1e39dc2c) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:29.803843 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59588e2f-f083-4105-ad1f-d7c8348adff9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:29.805314 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59588e2f-f083-4105-ad1f-d7c8348adff9 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:29.809740 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e0047-b6af-476f-af14-38750ed678fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:29.811736 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e0047-b6af-476f-af14-38750ed678fa None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:29.812728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 222/429] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Aug 3 18:37:29 2022] GET /identity/v3/role_assignments?group.id=7315f7c7db09430a962b9b45c222cfd1&scope.project.id=42291678aa1842ff9db31b9d1e39dc2c&role.id=f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:29.813672 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59588e2f-f083-4105-ad1f-d7c8348adff9 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:29.846747 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:29.948872 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 208/430] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:29.958444 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e0047-b6af-476f-af14-38750ed678fa None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:30.039061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59588e2f-f083-4105-ad1f-d7c8348adff9 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:30.092389 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903de096-c2ec-4eca-bb73-526abcb4b00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:30.093042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903de096-c2ec-4eca-bb73-526abcb4b00a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:30.093325 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903de096-c2ec-4eca-bb73-526abcb4b00a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:30.093592 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903de096-c2ec-4eca-bb73-526abcb4b00a None admin] PATH_INFO: `/v3/roles/member` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:30.107028 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903de096-c2ec-4eca-bb73-526abcb4b00a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-903de096-c2ec-4eca-bb73-526abcb4b00a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:30.113781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:30.114728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:30.115530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:30.115530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:30.115530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:30.115530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:30.115530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:30.115530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:30.115530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:30.115530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:30.115530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:30.115530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:30.115530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 18:37:30.115530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:30.115530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 223/431] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:30.124748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903de096-c2ec-4eca-bb73-526abcb4b00a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:30.137755 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e0047-b6af-476f-af14-38750ed678fa None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:30.155541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:30.174046 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:30.187899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66758ea-179f-460f-852d-d1a7c23091c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:30.188908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66758ea-179f-460f-852d-d1a7c23091c9 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:30.189390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66758ea-179f-460f-852d-d1a7c23091c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:30.189890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66758ea-179f-460f-852d-d1a7c23091c9 None admin] PATH_INFO: `/v3/projects/42291678aa1842ff9db31b9d1e39dc2c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:30.192779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66758ea-179f-460f-852d-d1a7c23091c9 None admin] RBAC: Authorizing `identity:get_project(project_id=42291678aa1842ff9db31b9d1e39dc2c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:30.194624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66758ea-179f-460f-852d-d1a7c23091c9 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:30.197083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 209/432] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:29 2022] GET /identity/v3/projects/42291678aa1842ff9db31b9d1e39dc2c => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:30.209547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66758ea-179f-460f-852d-d1a7c23091c9 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:30.263092 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903de096-c2ec-4eca-bb73-526abcb4b00a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:30.304145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e440b-f4c0-4a17-bd1e-8cc6d0a862ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:30.304784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e440b-f4c0-4a17-bd1e-8cc6d0a862ff None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:30.305069 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e440b-f4c0-4a17-bd1e-8cc6d0a862ff None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:30.305340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e440b-f4c0-4a17-bd1e-8cc6d0a862ff None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:30.307479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e440b-f4c0-4a17-bd1e-8cc6d0a862ff None admin] RBAC: Adding query filter params (name=member) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:30.307794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e440b-f4c0-4a17-bd1e-8cc6d0a862ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:30.311179 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e440b-f4c0-4a17-bd1e-8cc6d0a862ff None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:30.321695 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 224/433] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:37:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:30.323129 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c82233-5773-4bff-aa38-724a93321910 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:30.323413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c82233-5773-4bff-aa38-724a93321910 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:30.323684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c82233-5773-4bff-aa38-724a93321910 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:30.325123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 225/434] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:30.325997 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b88d00-7e29-4c3f-84ea-7c8c3e72fb8c None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:30.326483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b88d00-7e29-4c3f-84ea-7c8c3e72fb8c None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:30.326585 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b88d00-7e29-4c3f-84ea-7c8c3e72fb8c None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:30.327854 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 226/435] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:30.328447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b88d00-7e29-4c3f-84ea-7c8c3e72fb8c None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:30.343728 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66758ea-179f-460f-852d-d1a7c23091c9 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:30.387538 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e56e3e-da4b-4621-84f6-114e9fd4f9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:30.388403 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e56e3e-da4b-4621-84f6-114e9fd4f9b2 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:30.388698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e56e3e-da4b-4621-84f6-114e9fd4f9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:30.388978 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e56e3e-da4b-4621-84f6-114e9fd4f9b2 None admin] PATH_INFO: `/v3/groups/7315f7c7db09430a962b9b45c222cfd1` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:30.390834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e56e3e-da4b-4621-84f6-114e9fd4f9b2 None admin] RBAC: Authorizing `identity:get_group(group_id=7315f7c7db09430a962b9b45c222cfd1)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:30.393463 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e56e3e-da4b-4621-84f6-114e9fd4f9b2 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:30.395279 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 210/436] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Aug 3 18:37:30 2022] GET /identity/v3/groups/7315f7c7db09430a962b9b45c222cfd1 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:30.397080 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565142b-4cc5-4fc1-98ac-df56ad147120 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:30.397080 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565142b-4cc5-4fc1-98ac-df56ad147120 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:30.397301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565142b-4cc5-4fc1-98ac-df56ad147120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:30.417654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-2565142b-4cc5-4fc1-98ac-df56ad147120 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:30.418597 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-2565142b-4cc5-4fc1-98ac-df56ad147120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:30.433493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:30.455644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b88d00-7e29-4c3f-84ea-7c8c3e72fb8c None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:30.482421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:30.490918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa134f38-26e0-4a04-afc5-6a0ef7906484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:30.491549 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134f38-26e0-4a04-afc5-6a0ef7906484 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:30.491834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134f38-26e0-4a04-afc5-6a0ef7906484 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:30.492112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134f38-26e0-4a04-afc5-6a0ef7906484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:30.495016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 211/437] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:30.496443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9fa08-1681-42f9-b83f-c0145bb32a60 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:30.496820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9fa08-1681-42f9-b83f-c0145bb32a60 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:30.496920 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9fa08-1681-42f9-b83f-c0145bb32a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:30.520790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-64f9fa08-1681-42f9-b83f-c0145bb32a60 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:30.522068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-64f9fa08-1681-42f9-b83f-c0145bb32a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:30.565365 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa134f38-26e0-4a04-afc5-6a0ef7906484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:30.566735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa134f38-26e0-4a04-afc5-6a0ef7906484 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:30.591186 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 212/438] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:30.592057 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9fa08-1681-42f9-b83f-c0145bb32a60 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:30.652983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:30.673391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 227/439] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:30.674220 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa134f38-26e0-4a04-afc5-6a0ef7906484 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:30.694878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9fa08-1681-42f9-b83f-c0145bb32a60 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:30.715549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:30.730953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a311c99-997d-48f5-9008-9f69de480092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:30.731610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a311c99-997d-48f5-9008-9f69de480092 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:30.731890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a311c99-997d-48f5-9008-9f69de480092 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:30.732169 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a311c99-997d-48f5-9008-9f69de480092 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:30.733008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a311c99-997d-48f5-9008-9f69de480092 None admin] RBAC: Adding query filter params (group.id=7315f7c7db09430a962b9b45c222cfd1, role.id=548e622a9ae745d99d4cd34e68c1652c, scope.project.id=42291678aa1842ff9db31b9d1e39dc2c) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:30.733461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a311c99-997d-48f5-9008-9f69de480092 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:30.734797 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a311c99-997d-48f5-9008-9f69de480092 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:30.740484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 213/440] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Aug 3 18:37:30 2022] GET /identity/v3/role_assignments?group.id=7315f7c7db09430a962b9b45c222cfd1&scope.project.id=42291678aa1842ff9db31b9d1e39dc2c&role.id=548e622a9ae745d99d4cd34e68c1652c => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:30.741256 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a311c99-997d-48f5-9008-9f69de480092 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:30.789633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa134f38-26e0-4a04-afc5-6a0ef7906484 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:30.833078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad9cbf-0f23-4cda-8e77-751308b56400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:30.833642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad9cbf-0f23-4cda-8e77-751308b56400 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:30.833922 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad9cbf-0f23-4cda-8e77-751308b56400 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:30.834198 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad9cbf-0f23-4cda-8e77-751308b56400 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:30.847147 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad9cbf-0f23-4cda-8e77-751308b56400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-b2ad9cbf-0f23-4cda-8e77-751308b56400 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:30.851983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:30.854035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:30.855117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:30.855117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:30.855117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:30.855117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:30.855117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:30.855117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:30.855117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:30.855117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:30.855117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:30.855117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:30.855117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:37:30.855117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:30.855117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 228/441] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:37:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:30.855117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad9cbf-0f23-4cda-8e77-751308b56400 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:30.871276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a311c99-997d-48f5-9008-9f69de480092 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:30.907388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:30.915272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb23589-7fcf-4832-b6d1-ffa0150dd482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:30.917119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb23589-7fcf-4832-b6d1-ffa0150dd482 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:30.917409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb23589-7fcf-4832-b6d1-ffa0150dd482 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:30.917684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb23589-7fcf-4832-b6d1-ffa0150dd482 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:30.933445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb23589-7fcf-4832-b6d1-ffa0150dd482 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-cdb23589-7fcf-4832-b6d1-ffa0150dd482 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:30.938936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:30.939852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:30.944135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:30.944135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:30.944135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:30.944135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:30.944135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:30.944135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:30.944135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:30.944135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:30.944135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:30.944135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:30.944135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:30.944135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:30.944135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 214/442] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:37:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:30.944135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb23589-7fcf-4832-b6d1-ffa0150dd482 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:30.980808 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad9cbf-0f23-4cda-8e77-751308b56400 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:30.997195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:31.023129 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb7e21-c51d-4f57-87c3-d20a596b8cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:31.023779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb7e21-c51d-4f57-87c3-d20a596b8cdf None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:31.024065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb7e21-c51d-4f57-87c3-d20a596b8cdf None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:31.024391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb7e21-c51d-4f57-87c3-d20a596b8cdf None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:31.025789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb7e21-c51d-4f57-87c3-d20a596b8cdf None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:31.027265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb7e21-c51d-4f57-87c3-d20a596b8cdf None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:31.029294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 229/443] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:30 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:31.030121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb7e21-c51d-4f57-87c3-d20a596b8cdf None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:31.073834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb23589-7fcf-4832-b6d1-ffa0150dd482 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:31.115095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b763fd-e187-416f-95e0-8c779e8e69e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:31.116034 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b763fd-e187-416f-95e0-8c779e8e69e7 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:31.116409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b763fd-e187-416f-95e0-8c779e8e69e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:31.116719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b763fd-e187-416f-95e0-8c779e8e69e7 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:31.117550 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b763fd-e187-416f-95e0-8c779e8e69e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:31.117936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b763fd-e187-416f-95e0-8c779e8e69e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:31.120341 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b763fd-e187-416f-95e0-8c779e8e69e7 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:31.129164 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 215/444] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:31.129906 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b763fd-e187-416f-95e0-8c779e8e69e7 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:31.154235 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb7e21-c51d-4f57-87c3-d20a596b8cdf None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:31.164414 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:31.187752 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9ce04-edb7-47ff-8122-7203b473642c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:31.188400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9ce04-edb7-47ff-8122-7203b473642c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:31.188684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9ce04-edb7-47ff-8122-7203b473642c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:31.188959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9ce04-edb7-47ff-8122-7203b473642c None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:31.189677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9ce04-edb7-47ff-8122-7203b473642c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:31.189980 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9ce04-edb7-47ff-8122-7203b473642c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:31.191349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9ce04-edb7-47ff-8122-7203b473642c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:31.199726 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 230/445] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:31.200559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9ce04-edb7-47ff-8122-7203b473642c None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:31.232103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b763fd-e187-416f-95e0-8c779e8e69e7 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:31.238960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:31.268809 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a5cab-d1c8-4d30-bf83-c62b5b9795db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:31.269668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a5cab-d1c8-4d30-bf83-c62b5b9795db None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:31.269894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a5cab-d1c8-4d30-bf83-c62b5b9795db None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:31.270166 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a5cab-d1c8-4d30-bf83-c62b5b9795db None admin] PATH_INFO: `/v3/groups/7315f7c7db09430a962b9b45c222cfd1` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:31.271349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a5cab-d1c8-4d30-bf83-c62b5b9795db None admin] RBAC: Authorizing `identity:get_group(group_id=7315f7c7db09430a962b9b45c222cfd1)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:31.272816 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a5cab-d1c8-4d30-bf83-c62b5b9795db None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:31.275221 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 216/446] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Aug 3 18:37:31 2022] GET /identity/v3/groups/7315f7c7db09430a962b9b45c222cfd1 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:31.276004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a5cab-d1c8-4d30-bf83-c62b5b9795db None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:31.303633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9ce04-edb7-47ff-8122-7203b473642c None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:31.338918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32b294-0081-46fe-8ac3-62a01268b4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:31.339539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32b294-0081-46fe-8ac3-62a01268b4ac None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:31.339824 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32b294-0081-46fe-8ac3-62a01268b4ac None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:31.340106 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32b294-0081-46fe-8ac3-62a01268b4ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:31.376076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a5cab-d1c8-4d30-bf83-c62b5b9795db None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:31.382182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:31.410431 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32b294-0081-46fe-8ac3-62a01268b4ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:31.410883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24159d5e-f93f-4016-a77a-2a56e828aa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:31.411510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24159d5e-f93f-4016-a77a-2a56e828aa58 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:31.411795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24159d5e-f93f-4016-a77a-2a56e828aa58 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:31.411891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32b294-0081-46fe-8ac3-62a01268b4ac None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:31.412074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24159d5e-f93f-4016-a77a-2a56e828aa58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:31.456801 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:31.482146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24159d5e-f93f-4016-a77a-2a56e828aa58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:31.483803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24159d5e-f93f-4016-a77a-2a56e828aa58 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:31.515091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 231/447] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:31.520486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32b294-0081-46fe-8ac3-62a01268b4ac None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:31.588370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:31.590326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 217/448] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:31.591246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24159d5e-f93f-4016-a77a-2a56e828aa58 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:31.619865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32b294-0081-46fe-8ac3-62a01268b4ac None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:31.656108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c97702-82dd-4302-bf01-f9c4fc9a7640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:31.656672 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c97702-82dd-4302-bf01-f9c4fc9a7640 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:31.656952 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c97702-82dd-4302-bf01-f9c4fc9a7640 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:31.657224 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c97702-82dd-4302-bf01-f9c4fc9a7640 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:31.658021 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c97702-82dd-4302-bf01-f9c4fc9a7640 None admin] RBAC: Adding query filter params (group.id=7315f7c7db09430a962b9b45c222cfd1, role.id=f3b7ffe4d1f04af18f7da8d9a51031b3, scope.project.id=15984787df4448da8a1beaa8b7b15320) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:31.658343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c97702-82dd-4302-bf01-f9c4fc9a7640 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:31.659782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c97702-82dd-4302-bf01-f9c4fc9a7640 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:31.665646 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 232/449] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Aug 3 18:37:31 2022] GET /identity/v3/role_assignments?group.id=7315f7c7db09430a962b9b45c222cfd1&scope.project.id=15984787df4448da8a1beaa8b7b15320&role.id=f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:31.666498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c97702-82dd-4302-bf01-f9c4fc9a7640 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:31.671480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:31.702481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24159d5e-f93f-4016-a77a-2a56e828aa58 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:31.740347 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca058e-ded6-4d7d-8d04-306f2e54f123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:31.741024 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca058e-ded6-4d7d-8d04-306f2e54f123 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:31.741296 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca058e-ded6-4d7d-8d04-306f2e54f123 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:31.741574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca058e-ded6-4d7d-8d04-306f2e54f123 None admin] PATH_INFO: `/v3/projects/649429edefe44474b26fed9c93d5ec67` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:31.742805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca058e-ded6-4d7d-8d04-306f2e54f123 None admin] RBAC: Authorizing `identity:get_project(project_id=649429edefe44474b26fed9c93d5ec67)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:31.745038 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca058e-ded6-4d7d-8d04-306f2e54f123 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:31.751788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 218/450] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:31 2022] GET /identity/v3/projects/649429edefe44474b26fed9c93d5ec67 => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:31.752689 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca058e-ded6-4d7d-8d04-306f2e54f123 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:31.778731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c97702-82dd-4302-bf01-f9c4fc9a7640 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:31.819029 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce218e6-419f-4165-b6b5-8ea948cdf39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:31.819802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce218e6-419f-4165-b6b5-8ea948cdf39a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:31.820087 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce218e6-419f-4165-b6b5-8ea948cdf39a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:31.820554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce218e6-419f-4165-b6b5-8ea948cdf39a None admin] PATH_INFO: `/v3/projects/15984787df4448da8a1beaa8b7b15320` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:31.822252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce218e6-419f-4165-b6b5-8ea948cdf39a None admin] RBAC: Authorizing `identity:get_project(project_id=15984787df4448da8a1beaa8b7b15320)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:31.823883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce218e6-419f-4165-b6b5-8ea948cdf39a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:31.825702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 233/451] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:37:31 2022] GET /identity/v3/projects/15984787df4448da8a1beaa8b7b15320 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:31.831161 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce218e6-419f-4165-b6b5-8ea948cdf39a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:31.837956 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:31.876870 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca058e-ded6-4d7d-8d04-306f2e54f123 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:31.922666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcabfdc-0bf8-4eb2-a00a-1d632fc1c5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:31.923246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcabfdc-0bf8-4eb2-a00a-1d632fc1c5dd None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:31.923529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcabfdc-0bf8-4eb2-a00a-1d632fc1c5dd None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:31.923803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcabfdc-0bf8-4eb2-a00a-1d632fc1c5dd None admin] PATH_INFO: `/v3/groups/d01c88b6d1ed4973ac411d195d7c9e1b` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:31.925796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcabfdc-0bf8-4eb2-a00a-1d632fc1c5dd None admin] RBAC: Authorizing `identity:get_group(group_id=d01c88b6d1ed4973ac411d195d7c9e1b)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:31.926417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcabfdc-0bf8-4eb2-a00a-1d632fc1c5dd None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:31.928171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 219/452] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Aug 3 18:37:31 2022] GET /identity/v3/groups/d01c88b6d1ed4973ac411d195d7c9e1b => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:31.934031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcabfdc-0bf8-4eb2-a00a-1d632fc1c5dd None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:31.950172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:31.962895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce218e6-419f-4165-b6b5-8ea948cdf39a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:32.006817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8b6b7-f366-4207-8102-cff656082e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:32.007814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8b6b7-f366-4207-8102-cff656082e89 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:32.008109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8b6b7-f366-4207-8102-cff656082e89 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:32.008438 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8b6b7-f366-4207-8102-cff656082e89 None admin] PATH_INFO: `/v3/groups/7315f7c7db09430a962b9b45c222cfd1` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:32.010852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8b6b7-f366-4207-8102-cff656082e89 None admin] RBAC: Authorizing `identity:get_group(group_id=7315f7c7db09430a962b9b45c222cfd1)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:32.012344 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8b6b7-f366-4207-8102-cff656082e89 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:32.013993 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 234/453] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Aug 3 18:37:31 2022] GET /identity/v3/groups/7315f7c7db09430a962b9b45c222cfd1 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:32.019831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8b6b7-f366-4207-8102-cff656082e89 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:32.057796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcabfdc-0bf8-4eb2-a00a-1d632fc1c5dd None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:32.093414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633f672-3c26-4706-ab7c-a4f59cc50924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:32.094040 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633f672-3c26-4706-ab7c-a4f59cc50924 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:32.094319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633f672-3c26-4706-ab7c-a4f59cc50924 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:32.094587 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633f672-3c26-4706-ab7c-a4f59cc50924 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:32.095379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633f672-3c26-4706-ab7c-a4f59cc50924 None admin] RBAC: Adding query filter params (group.id=d01c88b6d1ed4973ac411d195d7c9e1b, role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=649429edefe44474b26fed9c93d5ec67) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:32.095690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633f672-3c26-4706-ab7c-a4f59cc50924 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:32.096916 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:32.097491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633f672-3c26-4706-ab7c-a4f59cc50924 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:32.103763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 220/454] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Aug 3 18:37:31 2022] GET /identity/v3/role_assignments?group.id=d01c88b6d1ed4973ac411d195d7c9e1b&scope.project.id=649429edefe44474b26fed9c93d5ec67&role.id=19557e941d6f475ba92cf862846f9872 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:32.128582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8b6b7-f366-4207-8102-cff656082e89 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:32.167346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612abebf-68ea-4c08-93fd-947fad6141a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:32.167976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612abebf-68ea-4c08-93fd-947fad6141a8 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:32.168261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612abebf-68ea-4c08-93fd-947fad6141a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:32.168563 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612abebf-68ea-4c08-93fd-947fad6141a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:32.169374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612abebf-68ea-4c08-93fd-947fad6141a8 None admin] RBAC: Adding query filter params (group.id=7315f7c7db09430a962b9b45c222cfd1, role.id=548e622a9ae745d99d4cd34e68c1652c, scope.project.id=15984787df4448da8a1beaa8b7b15320) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:32.169690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612abebf-68ea-4c08-93fd-947fad6141a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:32.171173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612abebf-68ea-4c08-93fd-947fad6141a8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:32.188433 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 235/455] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Aug 3 18:37:32 2022] GET /identity/v3/role_assignments?group.id=7315f7c7db09430a962b9b45c222cfd1&scope.project.id=15984787df4448da8a1beaa8b7b15320&role.id=548e622a9ae745d99d4cd34e68c1652c => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:34.687235 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9444c-0558-4ad1-9989-e29184404d37 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:34.687767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9444c-0558-4ad1-9989-e29184404d37 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:34.687767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9444c-0558-4ad1-9989-e29184404d37 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:34.689482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 221/456] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:34.694807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c5419-3665-4c4c-99a1-4f2f65804ca8 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:34.695226 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c5419-3665-4c4c-99a1-4f2f65804ca8 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:34.695390 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c5419-3665-4c4c-99a1-4f2f65804ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:34.724186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-308c5419-3665-4c4c-99a1-4f2f65804ca8 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:34.725411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-308c5419-3665-4c4c-99a1-4f2f65804ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:34.821266 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 236/457] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:34.844841 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9444c-0558-4ad1-9989-e29184404d37 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:34.969588 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7109b-a73e-41a4-a057-3fa1ccf2f300 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:34.969907 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7109b-a73e-41a4-a057-3fa1ccf2f300 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:34.970146 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7109b-a73e-41a4-a057-3fa1ccf2f300 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:34.971500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 237/458] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:37:34.972159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:34.977089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d259f3a-a597-444b-8f4b-10bb232daa9b None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:34.977891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d259f3a-a597-444b-8f4b-10bb232daa9b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:34.978178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d259f3a-a597-444b-8f4b-10bb232daa9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:34.998229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-4d259f3a-a597-444b-8f4b-10bb232daa9b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:34.999413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4d259f3a-a597-444b-8f4b-10bb232daa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:35.029553 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9444c-0558-4ad1-9989-e29184404d37 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:35.076414 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 238/459] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:35.076807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:35.113064 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452280e-06ec-48e0-b547-3f5d6d21c2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:35.113730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452280e-06ec-48e0-b547-3f5d6d21c2fc None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:35.114011 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452280e-06ec-48e0-b547-3f5d6d21c2fc None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:35.114284 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452280e-06ec-48e0-b547-3f5d6d21c2fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:35.123479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d259f3a-a597-444b-8f4b-10bb232daa9b None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:35.133445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452280e-06ec-48e0-b547-3f5d6d21c2fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:35.136109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452280e-06ec-48e0-b547-3f5d6d21c2fc None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:35.166407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-8452280e-06ec-48e0-b547-3f5d6d21c2fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:35.166407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:35.166407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:35.166407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:35.166407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:35.166407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:35.166407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:35.166407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:35.166407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:35.166407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:35.166407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:35.166407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:35.166407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:35.166407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:35.166407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:35.167482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:35.168368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:35.168368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:35.168368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:35.168368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:35.173347 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 222/460] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:35.177473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452280e-06ec-48e0-b547-3f5d6d21c2fc None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:35.274102 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d259f3a-a597-444b-8f4b-10bb232daa9b None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:35.310975 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d83ee3-8504-48d5-ba0e-3d2bbfd62825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:35.311629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d83ee3-8504-48d5-ba0e-3d2bbfd62825 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:35.311959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d83ee3-8504-48d5-ba0e-3d2bbfd62825 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:35.312248 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d83ee3-8504-48d5-ba0e-3d2bbfd62825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:35.323688 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d83ee3-8504-48d5-ba0e-3d2bbfd62825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:35.325203 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d83ee3-8504-48d5-ba0e-3d2bbfd62825 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:35.329899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452280e-06ec-48e0-b547-3f5d6d21c2fc None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:35.346617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-e9d83ee3-8504-48d5-ba0e-3d2bbfd62825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:35.346617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:35.346617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:35.346617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:35.346617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:35.346617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:35.346617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:35.346617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:35.346617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:35.346617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:35.346617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:35.346617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:35.346617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:35.346617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:35.346617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:35.347519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:35.348425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:35.348425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:35.348425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:35.348425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:35.349720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 239/461] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:35.350746 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c0825-526b-4d1f-9c7c-4db2f8430761 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:35.351032 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c0825-526b-4d1f-9c7c-4db2f8430761 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:35.351308 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c0825-526b-4d1f-9c7c-4db2f8430761 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:35.356001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 240/462] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:35.357496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c0825-526b-4d1f-9c7c-4db2f8430761 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:35.365812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:35.375853 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fcc3b-f0a1-45f2-9950-03d8b58bb15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:35.376757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fcc3b-f0a1-45f2-9950-03d8b58bb15f None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:35.377197 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fcc3b-f0a1-45f2-9950-03d8b58bb15f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:35.377607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fcc3b-f0a1-45f2-9950-03d8b58bb15f None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:35.378446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fcc3b-f0a1-45f2-9950-03d8b58bb15f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:35.379154 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fcc3b-f0a1-45f2-9950-03d8b58bb15f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:35.380563 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fcc3b-f0a1-45f2-9950-03d8b58bb15f None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:35.394456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 223/463] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:35.395955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11303772-25e4-45e1-987a-2cf116315b32 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:35.396232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11303772-25e4-45e1-987a-2cf116315b32 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:35.396533 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11303772-25e4-45e1-987a-2cf116315b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:35.417045 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-11303772-25e4-45e1-987a-2cf116315b32 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:35.418336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-11303772-25e4-45e1-987a-2cf116315b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:35.420649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:35.488141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c0825-526b-4d1f-9c7c-4db2f8430761 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:35.501045 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 224/464] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:35.502493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11303772-25e4-45e1-987a-2cf116315b32 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:35.529874 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8008e26-6450-4094-926f-b6491858f66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:35.530411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8008e26-6450-4094-926f-b6491858f66e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:35.530691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8008e26-6450-4094-926f-b6491858f66e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:35.530970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8008e26-6450-4094-926f-b6491858f66e None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:35.531655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8008e26-6450-4094-926f-b6491858f66e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:35.531959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8008e26-6450-4094-926f-b6491858f66e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:35.533885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8008e26-6450-4094-926f-b6491858f66e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:35.547690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 241/465] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:35.552216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf44dee-6118-4715-82dd-4fbac0c00154 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:35.552216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf44dee-6118-4715-82dd-4fbac0c00154 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:35.552216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf44dee-6118-4715-82dd-4fbac0c00154 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:35.552216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 242/466] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:35.552216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf44dee-6118-4715-82dd-4fbac0c00154 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:35.624300 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11303772-25e4-45e1-987a-2cf116315b32 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:35.645972 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:35.670061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fac8a8-76e8-4df2-8113-a41dcf21ff37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:35.671375 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fac8a8-76e8-4df2-8113-a41dcf21ff37 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:35.671674 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fac8a8-76e8-4df2-8113-a41dcf21ff37 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:35.671955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fac8a8-76e8-4df2-8113-a41dcf21ff37 None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:35.672724 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fac8a8-76e8-4df2-8113-a41dcf21ff37 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:35.674012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf44dee-6118-4715-82dd-4fbac0c00154 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:35.674257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fac8a8-76e8-4df2-8113-a41dcf21ff37 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:35.684702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:35.709539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 225/467] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:37:35 2022] POST /identity/v3/users => generated 246 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:37:35.709539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fac8a8-76e8-4df2-8113-a41dcf21ff37 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:35.712511 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f21f9f-faba-4205-a2fc-4f6f074485bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:35.713223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f21f9f-faba-4205-a2fc-4f6f074485bf None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:35.713508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f21f9f-faba-4205-a2fc-4f6f074485bf None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:35.713911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f21f9f-faba-4205-a2fc-4f6f074485bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:35.729895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f21f9f-faba-4205-a2fc-4f6f074485bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:35.731429 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f21f9f-faba-4205-a2fc-4f6f074485bf None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:35.749323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-a9f21f9f-faba-4205-a2fc-4f6f074485bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:35.749323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:35.749323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:35.749323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:35.749323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:35.749323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:35.749323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:35.749323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:35.749323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:35.749323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:35.749323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:35.749323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:35.749323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:35.749323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:35.749323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:35.750282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:35.751144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:35.751144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:35.751144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:35.751144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:35.755886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 243/468] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:35.757381 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c797c3-8c05-4369-83d4-54563cb8f177 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:35.757667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c797c3-8c05-4369-83d4-54563cb8f177 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:35.757945 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c797c3-8c05-4369-83d4-54563cb8f177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:35.778871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-f8c797c3-8c05-4369-83d4-54563cb8f177 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:35.780158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f8c797c3-8c05-4369-83d4-54563cb8f177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:35.854236 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fac8a8-76e8-4df2-8113-a41dcf21ff37 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:35.870934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 244/469] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:35.871717 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c797c3-8c05-4369-83d4-54563cb8f177 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:35.900698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d8e56-6122-494a-86f5-a146ce74e053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:35.901343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d8e56-6122-494a-86f5-a146ce74e053 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:35.901705 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d8e56-6122-494a-86f5-a146ce74e053 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:35.901889 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d8e56-6122-494a-86f5-a146ce74e053 None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:35.902580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d8e56-6122-494a-86f5-a146ce74e053 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:35.904094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d8e56-6122-494a-86f5-a146ce74e053 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:35.933764 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 226/470] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:37:35 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:37:35.934689 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d8e56-6122-494a-86f5-a146ce74e053 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:35.972519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:36.015030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:36.031419 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c797c3-8c05-4369-83d4-54563cb8f177 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:36.079879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d8e56-6122-494a-86f5-a146ce74e053 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:36.082412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116d10d-c4a7-457c-83e0-3bc1faf89fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:36.084275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116d10d-c4a7-457c-83e0-3bc1faf89fbd None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:36.084762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116d10d-c4a7-457c-83e0-3bc1faf89fbd None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:36.085203 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116d10d-c4a7-457c-83e0-3bc1faf89fbd None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:36.086084 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116d10d-c4a7-457c-83e0-3bc1faf89fbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:36.086533 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116d10d-c4a7-457c-83e0-3bc1faf89fbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:36.088960 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116d10d-c4a7-457c-83e0-3bc1faf89fbd None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:36.107112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 245/471] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:36.113550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116d10d-c4a7-457c-83e0-3bc1faf89fbd None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:36.135327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb896ef-ffba-4708-af4a-b8a0a9dcb96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:36.135992 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb896ef-ffba-4708-af4a-b8a0a9dcb96b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:36.136275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb896ef-ffba-4708-af4a-b8a0a9dcb96b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:36.136573 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb896ef-ffba-4708-af4a-b8a0a9dcb96b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:36.150617 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb896ef-ffba-4708-af4a-b8a0a9dcb96b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:36.152091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb896ef-ffba-4708-af4a-b8a0a9dcb96b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:36.170001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-0fb896ef-ffba-4708-af4a-b8a0a9dcb96b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:36.170001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:36.170001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:36.170001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:36.170001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:36.170001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:36.170001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:36.170001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:36.170001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:36.170001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:36.170001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:36.170001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:36.170001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:36.170001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:36.170001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:36.171091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:36.172087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:36.172087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:36.172087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:36.172087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:36.172087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 227/472] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:36.178468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb896ef-ffba-4708-af4a-b8a0a9dcb96b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:36.242059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116d10d-c4a7-457c-83e0-3bc1faf89fbd None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:36.271670 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:36.282676 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3dfde-72d6-4020-8e5e-d7065878b742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:36.283612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3dfde-72d6-4020-8e5e-d7065878b742 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:36.284107 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3dfde-72d6-4020-8e5e-d7065878b742 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:36.284600 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3dfde-72d6-4020-8e5e-d7065878b742 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:36.285478 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3dfde-72d6-4020-8e5e-d7065878b742 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:36.287251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3dfde-72d6-4020-8e5e-d7065878b742 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:36.297011 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb896ef-ffba-4708-af4a-b8a0a9dcb96b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:36.317702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 246/473] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:37:36 2022] POST /identity/v3/users => generated 248 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:37:36.341405 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf94a6-90a5-43b1-87eb-b2dc1e191e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:36.342057 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf94a6-90a5-43b1-87eb-b2dc1e191e03 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:36.342341 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf94a6-90a5-43b1-87eb-b2dc1e191e03 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:36.342618 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf94a6-90a5-43b1-87eb-b2dc1e191e03 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:36.343346 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf94a6-90a5-43b1-87eb-b2dc1e191e03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:36.343651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf94a6-90a5-43b1-87eb-b2dc1e191e03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:36.345105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf94a6-90a5-43b1-87eb-b2dc1e191e03 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:36.360409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 228/474] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:36.366239 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3dfde-72d6-4020-8e5e-d7065878b742 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:36.382774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:36.508307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3dfde-72d6-4020-8e5e-d7065878b742 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:36.510633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40801d-faaa-4241-a514-5074d478b574 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:36.510927 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40801d-faaa-4241-a514-5074d478b574 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:36.511203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40801d-faaa-4241-a514-5074d478b574 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:36.515147 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 229/475] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:36.519495 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6839f-6764-47d3-9d32-3c8b70d22563 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:36.519788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6839f-6764-47d3-9d32-3c8b70d22563 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:36.520072 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6839f-6764-47d3-9d32-3c8b70d22563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:36.546918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a34ff-94f0-4c30-b617-dc07829dd9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:36.547478 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a34ff-94f0-4c30-b617-dc07829dd9af None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:36.547762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a34ff-94f0-4c30-b617-dc07829dd9af None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:36.548040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a34ff-94f0-4c30-b617-dc07829dd9af None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:36.548819 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a34ff-94f0-4c30-b617-dc07829dd9af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:36.550268 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a34ff-94f0-4c30-b617-dc07829dd9af None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:36.550555 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-01e6839f-6764-47d3-9d32-3c8b70d22563 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:36.551751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-01e6839f-6764-47d3-9d32-3c8b70d22563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:36.580584 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 247/476] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:37:36 2022] POST /identity/v3/users => generated 249 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:37:36.648910 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 230/477] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:36.669480 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a34ff-94f0-4c30-b617-dc07829dd9af None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:36.758072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:36.800659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a34ff-94f0-4c30-b617-dc07829dd9af None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:36.837678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154a90b-8f72-4205-9056-97e80fc474a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:36.838321 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154a90b-8f72-4205-9056-97e80fc474a8 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:36.838602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154a90b-8f72-4205-9056-97e80fc474a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:36.838883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154a90b-8f72-4205-9056-97e80fc474a8 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:36.839612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154a90b-8f72-4205-9056-97e80fc474a8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:36.841030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154a90b-8f72-4205-9056-97e80fc474a8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:36.848785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.sql.core [None req-e154a90b-8f72-4205-9056-97e80fc474a8 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 03 18:37:36.848785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 03 18:37:36.848785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [parameters: {'id': '22506df225f843bd8dbfec59f9941978', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 03 18:37:36.848785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91895) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 03 18:37:36.865944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-e154a90b-8f72-4205-9056-97e80fc474a8 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 03 18:37:36.865944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:36.865944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 18:37:36.865944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 18:37:36.865944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 18:37:36.865944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 18:37:36.865944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 18:37:36.865944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 18:37:36.865944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 18:37:36.865944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application conn.query(q) Aug 03 18:37:36.865944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 18:37:36.865944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 18:37:36.865944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 18:37:36.865944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application result.read() Aug 03 18:37:36.865944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return ref.to_dict() Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 18:37:36.866875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application next(self.gen) Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application yield resource Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application next(self.gen) Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self.session.rollback() Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self.force_reraise() Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise self.value Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application session.commit() Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 03 18:37:36.868639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._prepare_impl() Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self.session.flush() Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._flush(objects) Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application compat.raise_( Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application flush_context.execute() Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rec.execute(self) Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 03 18:37:36.869624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 18:37:36.871444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application conn.query(q) Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application result.read() Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 18:37:36.873334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [parameters: {'id': '22506df225f843bd8dbfec59f9941978', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:36.875282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 03 18:37:36.878249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:36.880659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 248/478] 10.222.0.69 () {60 vars in 1131 bytes} [Wed Aug 3 18:37:36 2022] POST /identity/v3/roles => generated 144 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:36.883288 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6839f-6764-47d3-9d32-3c8b70d22563 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:36.962361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:36.996460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6839f-6764-47d3-9d32-3c8b70d22563 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:37.034426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aec3b9-3faa-4288-8ddc-cb086bc4fb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:37.035435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aec3b9-3faa-4288-8ddc-cb086bc4fb3b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:37.035940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aec3b9-3faa-4288-8ddc-cb086bc4fb3b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:37.036476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aec3b9-3faa-4288-8ddc-cb086bc4fb3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:37.048753 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aec3b9-3faa-4288-8ddc-cb086bc4fb3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-a2aec3b9-3faa-4288-8ddc-cb086bc4fb3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:37.054095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:37.054970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:37.056491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:37.056491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:37.056491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:37.056491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:37.056491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:37.056491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:37.056491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:37.056491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:37.056491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:37.056491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:37.056491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 18:37:37.056491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:37.059826 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 231/479] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:37:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:37.068338 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154a90b-8f72-4205-9056-97e80fc474a8 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:37.215229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154a90b-8f72-4205-9056-97e80fc474a8 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:37.252809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:37.263223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc28f0-9cf7-46ae-b908-abb91006e332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:37.264408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc28f0-9cf7-46ae-b908-abb91006e332 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:37.264986 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc28f0-9cf7-46ae-b908-abb91006e332 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:37.265626 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc28f0-9cf7-46ae-b908-abb91006e332 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:37.266595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc28f0-9cf7-46ae-b908-abb91006e332 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:37.267298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc28f0-9cf7-46ae-b908-abb91006e332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:37.268745 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc28f0-9cf7-46ae-b908-abb91006e332 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:37.280354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 249/480] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:38.356890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5ef7a-6656-4477-91a9-daf8fbe281d2 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:38.357414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5ef7a-6656-4477-91a9-daf8fbe281d2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:38.357684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5ef7a-6656-4477-91a9-daf8fbe281d2 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:38.358887 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 232/481] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:38.364267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd2021-8d4f-4dfb-adb6-a19decf24a0a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:38.364703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd2021-8d4f-4dfb-adb6-a19decf24a0a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:38.364869 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd2021-8d4f-4dfb-adb6-a19decf24a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:38.389412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-63dd2021-8d4f-4dfb-adb6-a19decf24a0a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:38.390699 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-63dd2021-8d4f-4dfb-adb6-a19decf24a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:38.486348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 250/482] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:38.529367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5ef7a-6656-4477-91a9-daf8fbe281d2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:38.642160 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5ef7a-6656-4477-91a9-daf8fbe281d2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:38.673741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:38.680049 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffef42-4b4e-43e7-a2e3-4383a2873ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:38.681258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffef42-4b4e-43e7-a2e3-4383a2873ca6 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:38.681474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffef42-4b4e-43e7-a2e3-4383a2873ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:38.681706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffef42-4b4e-43e7-a2e3-4383a2873ca6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:38.692960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffef42-4b4e-43e7-a2e3-4383a2873ca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-ebffef42-4b4e-43e7-a2e3-4383a2873ca6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:38.699239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:38.701350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:38.703263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:38.703263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:38.703263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:38.703263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:38.703263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:38.703263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:38.703263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:38.703263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:38.703263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:38.703263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:38.703263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:38.703263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:38.708316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 233/483] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:38.708503 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd2021-8d4f-4dfb-adb6-a19decf24a0a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:38.737892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47238c1-316d-411c-a868-82f336604f16 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:38.738226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47238c1-316d-411c-a868-82f336604f16 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:38.738521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47238c1-316d-411c-a868-82f336604f16 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:38.740019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 234/484] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:37:38.745235 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9f56-4ea1-424a-a9c8-d6397e99289a None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:38.745520 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9f56-4ea1-424a-a9c8-d6397e99289a None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:38.745796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9f56-4ea1-424a-a9c8-d6397e99289a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:38.775074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-d7fc9f56-4ea1-424a-a9c8-d6397e99289a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:38.776396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d7fc9f56-4ea1-424a-a9c8-d6397e99289a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:38.893012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd2021-8d4f-4dfb-adb6-a19decf24a0a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:38.926110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:38.927111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 235/485] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:38.928401 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4f429-bc36-4f8b-a890-3b2438d384d1 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:38.928687 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4f429-bc36-4f8b-a890-3b2438d384d1 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:38.928958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4f429-bc36-4f8b-a890-3b2438d384d1 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:38.934103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 236/486] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:38.945371 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da019f-11e4-4c4a-820d-eff57f1066ab None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:38.945790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da019f-11e4-4c4a-820d-eff57f1066ab None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:38.946895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da019f-11e4-4c4a-820d-eff57f1066ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:38.961281 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfb111-0975-4ffd-accd-76ce62f48785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:38.962481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfb111-0975-4ffd-accd-76ce62f48785 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:38.962796 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfb111-0975-4ffd-accd-76ce62f48785 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:38.963170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfb111-0975-4ffd-accd-76ce62f48785 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:38.963996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfb111-0975-4ffd-accd-76ce62f48785 None admin] RBAC: Adding query filter params (name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:38.964440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfb111-0975-4ffd-accd-76ce62f48785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:38.965924 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfb111-0975-4ffd-accd-76ce62f48785 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:38.968508 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-01da019f-11e4-4c4a-820d-eff57f1066ab None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:38.969783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-01da019f-11e4-4c4a-820d-eff57f1066ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:38.974786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 251/487] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:37:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:38.975685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfb111-0975-4ffd-accd-76ce62f48785 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:39.047242 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 237/488] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:39.048182 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da019f-11e4-4c4a-820d-eff57f1066ab None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:39.102528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:39.129237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfb111-0975-4ffd-accd-76ce62f48785 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:39.160380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da019f-11e4-4c4a-820d-eff57f1066ab None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:39.166001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d51fe-dc3e-46ac-80de-d68017915ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:39.166701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d51fe-dc3e-46ac-80de-d68017915ea4 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:39.167064 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d51fe-dc3e-46ac-80de-d68017915ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:39.167538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d51fe-dc3e-46ac-80de-d68017915ea4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:39.178561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d51fe-dc3e-46ac-80de-d68017915ea4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-2b5d51fe-dc3e-46ac-80de-d68017915ea4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:39.185008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:39.187127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:39.189127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:39.189127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:39.189127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:39.189127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:39.189127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:39.189127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:39.189127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:39.189127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:39.189127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:39.189127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:39.189127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:39.189127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:39.189127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 252/489] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:39.189127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d51fe-dc3e-46ac-80de-d68017915ea4 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:39.204197 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28828f-8e9c-44c3-b529-4bbf3afdb431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:39.205004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28828f-8e9c-44c3-b529-4bbf3afdb431 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:39.205284 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28828f-8e9c-44c3-b529-4bbf3afdb431 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:39.205595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28828f-8e9c-44c3-b529-4bbf3afdb431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:39.216880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28828f-8e9c-44c3-b529-4bbf3afdb431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:39.218530 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28828f-8e9c-44c3-b529-4bbf3afdb431 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:39.232039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:39.236622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-8a28828f-8e9c-44c3-b529-4bbf3afdb431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:39.236622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:39.236622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:39.236622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:39.236622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:39.236622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:39.236622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:39.236622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:39.236622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:39.236622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:39.236622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:39.236622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:39.236622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:39.236622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:39.236622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:39.237713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:39.238675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:39.238675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:39.238675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:39.238675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:39.240544 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 238/490] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:39.241464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28828f-8e9c-44c3-b529-4bbf3afdb431 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:39.303543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d51fe-dc3e-46ac-80de-d68017915ea4 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:39.340000 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76803e5a-fb23-4e1b-bcc6-96fd5d9ffb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:39.340685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76803e5a-fb23-4e1b-bcc6-96fd5d9ffb1a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:39.341222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76803e5a-fb23-4e1b-bcc6-96fd5d9ffb1a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:39.341222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76803e5a-fb23-4e1b-bcc6-96fd5d9ffb1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:39.352846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76803e5a-fb23-4e1b-bcc6-96fd5d9ffb1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-76803e5a-fb23-4e1b-bcc6-96fd5d9ffb1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:39.358668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:39.361610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:39.363852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:39.363852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:39.363852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:39.363852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:39.363852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:39.363852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:39.363852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:39.363852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:39.363852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:39.363852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:39.363852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:39.363852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:39.367664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28828f-8e9c-44c3-b529-4bbf3afdb431 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:39.369346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 253/491] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:39.369346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76803e5a-fb23-4e1b-bcc6-96fd5d9ffb1a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:39.370654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:39.403546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6ca2e-1886-4973-88d3-f4cc707b2874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:39.404171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6ca2e-1886-4973-88d3-f4cc707b2874 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:39.404484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6ca2e-1886-4973-88d3-f4cc707b2874 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:39.404761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6ca2e-1886-4973-88d3-f4cc707b2874 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:39.405547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6ca2e-1886-4973-88d3-f4cc707b2874 None admin] RBAC: Adding query filter params (name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:39.405855 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6ca2e-1886-4973-88d3-f4cc707b2874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:39.407409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6ca2e-1886-4973-88d3-f4cc707b2874 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:39.416869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 239/492] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:37:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:39.418034 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf1ad9-90b6-4eec-a8a3-34242c00062f None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:39.418318 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf1ad9-90b6-4eec-a8a3-34242c00062f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:39.418586 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf1ad9-90b6-4eec-a8a3-34242c00062f None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:39.419785 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 240/493] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:39.420491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf1ad9-90b6-4eec-a8a3-34242c00062f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:39.481447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:39.485066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76803e5a-fb23-4e1b-bcc6-96fd5d9ffb1a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:39.522388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32f2cf-278f-472e-bbd4-b9c5d263ddff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:39.523015 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32f2cf-278f-472e-bbd4-b9c5d263ddff None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:39.523302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32f2cf-278f-472e-bbd4-b9c5d263ddff None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:39.523579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32f2cf-278f-472e-bbd4-b9c5d263ddff None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:39.524466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32f2cf-278f-472e-bbd4-b9c5d263ddff None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:39.524886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32f2cf-278f-472e-bbd4-b9c5d263ddff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:39.526373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32f2cf-278f-472e-bbd4-b9c5d263ddff None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:39.534031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf1ad9-90b6-4eec-a8a3-34242c00062f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:39.540814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 254/494] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:39.546353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32f2cf-278f-472e-bbd4-b9c5d263ddff None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:39.570958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c02ed4-a6db-4f67-9b97-92b0e67676a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:39.571579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c02ed4-a6db-4f67-9b97-92b0e67676a1 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:39.572170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c02ed4-a6db-4f67-9b97-92b0e67676a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:39.572489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c02ed4-a6db-4f67-9b97-92b0e67676a1 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:39.573213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c02ed4-a6db-4f67-9b97-92b0e67676a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:39.573521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c02ed4-a6db-4f67-9b97-92b0e67676a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:39.574889 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c02ed4-a6db-4f67-9b97-92b0e67676a1 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:39.587367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 241/495] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:37:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:39.587367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abdd57-4d0a-41c5-a79c-6019a427754f None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:39.587367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abdd57-4d0a-41c5-a79c-6019a427754f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:39.587367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abdd57-4d0a-41c5-a79c-6019a427754f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:39.603360 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-30abdd57-4d0a-41c5-a79c-6019a427754f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:39.604589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-30abdd57-4d0a-41c5-a79c-6019a427754f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:39.626005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:39.661770 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32f2cf-278f-472e-bbd4-b9c5d263ddff None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:39.684559 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 242/496] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:39.684559 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abdd57-4d0a-41c5-a79c-6019a427754f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:39.709197 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece661e-8f6e-4698-bb85-88ee28a995d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:39.709851 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece661e-8f6e-4698-bb85-88ee28a995d9 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:39.710134 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece661e-8f6e-4698-bb85-88ee28a995d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:39.710411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece661e-8f6e-4698-bb85-88ee28a995d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:39.722462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece661e-8f6e-4698-bb85-88ee28a995d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:39.723932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece661e-8f6e-4698-bb85-88ee28a995d9 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:39.741812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-bece661e-8f6e-4698-bb85-88ee28a995d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:39.741812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:39.741812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:39.741812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:39.741812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:39.741812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:39.741812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:39.741812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:39.741812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:39.741812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:39.741812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:39.741812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:39.741812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:39.741812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:39.741812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:39.742761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:39.743775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:39.743775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:39.743775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:39.743775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:39.748250 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 255/497] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:39.749170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece661e-8f6e-4698-bb85-88ee28a995d9 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:39.765666 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:39.798113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abdd57-4d0a-41c5-a79c-6019a427754f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:39.838508 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420201bb-238c-4748-bbb7-d082e8fdd05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:39.839157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420201bb-238c-4748-bbb7-d082e8fdd05a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:39.839455 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420201bb-238c-4748-bbb7-d082e8fdd05a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:39.839706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420201bb-238c-4748-bbb7-d082e8fdd05a None admin] PATH_INFO: `/v3/users/nova` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:39.848707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420201bb-238c-4748-bbb7-d082e8fdd05a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:39.850141 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420201bb-238c-4748-bbb7-d082e8fdd05a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-420201bb-238c-4748-bbb7-d082e8fdd05a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:39.865063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:39.866031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:39.866917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:39.866917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:39.866917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:39.866917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:39.866917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:39.866917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:39.866917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:39.866917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:39.866917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:37:39.866917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:39.866917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 243/498] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Aug 3 18:37:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:39.867493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420201bb-238c-4748-bbb7-d082e8fdd05a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:39.873217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece661e-8f6e-4698-bb85-88ee28a995d9 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:39.887561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:39.910450 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bd89e-f54f-4ccc-a0f1-38034e6994c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:39.911099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bd89e-f54f-4ccc-a0f1-38034e6994c1 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:39.911814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bd89e-f54f-4ccc-a0f1-38034e6994c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:39.912093 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bd89e-f54f-4ccc-a0f1-38034e6994c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:39.923705 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bd89e-f54f-4ccc-a0f1-38034e6994c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:39.925279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bd89e-f54f-4ccc-a0f1-38034e6994c1 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:39.941279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-e77bd89e-f54f-4ccc-a0f1-38034e6994c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:39.941279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:39.941279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:39.941279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:39.941279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:39.941279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:39.941279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:39.941279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:39.941279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:39.941279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:39.941279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:39.941279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:39.941279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:39.941279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:39.941279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:39.942307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:39.943352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:39.943352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:39.943352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:39.943352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:39.947253 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 256/499] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:39.948112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bd89e-f54f-4ccc-a0f1-38034e6994c1 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:39.987348 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420201bb-238c-4748-bbb7-d082e8fdd05a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:40.014830 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:40.024866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f8e62-201e-4284-a572-8fad1dbfb606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:40.025882 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f8e62-201e-4284-a572-8fad1dbfb606 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:40.026171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f8e62-201e-4284-a572-8fad1dbfb606 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:40.026449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f8e62-201e-4284-a572-8fad1dbfb606 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:40.037047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f8e62-201e-4284-a572-8fad1dbfb606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-570f8e62-201e-4284-a572-8fad1dbfb606 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:40.042551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:40.043454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:40.044251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:40.044251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:40.044251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:40.044251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:40.044251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:40.044251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:40.044251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:40.044251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:40.044251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:40.044251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:40.044251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:40.044251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:40.044251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 244/500] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:40.044985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f8e62-201e-4284-a572-8fad1dbfb606 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:40.058797 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bd89e-f54f-4ccc-a0f1-38034e6994c1 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:40.095280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1a3d6-b276-4536-847f-ad1043d2d101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:40.095893 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1a3d6-b276-4536-847f-ad1043d2d101 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:40.096173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1a3d6-b276-4536-847f-ad1043d2d101 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:40.096483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1a3d6-b276-4536-847f-ad1043d2d101 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:40.097175 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec1a3d6-b276-4536-847f-ad1043d2d101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:40.097479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec1a3d6-b276-4536-847f-ad1043d2d101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:40.098878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec1a3d6-b276-4536-847f-ad1043d2d101 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:40.110997 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 257/501] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:40.111793 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1a3d6-b276-4536-847f-ad1043d2d101 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:40.123270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:40.147456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f8e62-201e-4284-a572-8fad1dbfb606 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:40.182683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ad566-57eb-498f-8965-cfe41ffb1ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:40.183299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ad566-57eb-498f-8965-cfe41ffb1ce6 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:40.183583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ad566-57eb-498f-8965-cfe41ffb1ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:40.183860 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ad566-57eb-498f-8965-cfe41ffb1ce6 None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:40.184703 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ad566-57eb-498f-8965-cfe41ffb1ce6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:40.185012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ad566-57eb-498f-8965-cfe41ffb1ce6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:40.186501 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ad566-57eb-498f-8965-cfe41ffb1ce6 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:40.219440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1a3d6-b276-4536-847f-ad1043d2d101 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:40.224322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 245/502] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Aug 3 18:37:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:40.225579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a8236-8845-4011-93fc-c0cab0b47d03 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:40.225868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a8236-8845-4011-93fc-c0cab0b47d03 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:40.226146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a8236-8845-4011-93fc-c0cab0b47d03 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:40.228494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 246/503] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:40.228938 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a8236-8845-4011-93fc-c0cab0b47d03 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:40.244305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:40.255978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981b7b5-0527-44b2-b43c-7b7d4d89b885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:40.256627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981b7b5-0527-44b2-b43c-7b7d4d89b885 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:40.256922 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981b7b5-0527-44b2-b43c-7b7d4d89b885 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:40.257198 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981b7b5-0527-44b2-b43c-7b7d4d89b885 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:40.257897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981b7b5-0527-44b2-b43c-7b7d4d89b885 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:40.258204 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981b7b5-0527-44b2-b43c-7b7d4d89b885 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:40.260312 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981b7b5-0527-44b2-b43c-7b7d4d89b885 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:40.271331 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 258/504] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:40.272122 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981b7b5-0527-44b2-b43c-7b7d4d89b885 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:40.332185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a8236-8845-4011-93fc-c0cab0b47d03 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:40.366127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00c5fd-5bf3-4a28-925b-f7032ddb246a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:40.366745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00c5fd-5bf3-4a28-925b-f7032ddb246a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:40.367025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00c5fd-5bf3-4a28-925b-f7032ddb246a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:40.367302 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00c5fd-5bf3-4a28-925b-f7032ddb246a None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:40.368482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00c5fd-5bf3-4a28-925b-f7032ddb246a None admin] RBAC: Adding query filter params (name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:40.369030 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00c5fd-5bf3-4a28-925b-f7032ddb246a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:40.370264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00c5fd-5bf3-4a28-925b-f7032ddb246a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:40.371646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:40.376705 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981b7b5-0527-44b2-b43c-7b7d4d89b885 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:40.379442 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 247/505] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:37:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:40.380233 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00c5fd-5bf3-4a28-925b-f7032ddb246a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:40.412216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa56ad0-e7c7-4371-8fbf-6f8f2a0efbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:40.412892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa56ad0-e7c7-4371-8fbf-6f8f2a0efbc5 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:40.413171 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa56ad0-e7c7-4371-8fbf-6f8f2a0efbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:40.413443 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa56ad0-e7c7-4371-8fbf-6f8f2a0efbc5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:40.421727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa56ad0-e7c7-4371-8fbf-6f8f2a0efbc5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:40.423182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa56ad0-e7c7-4371-8fbf-6f8f2a0efbc5 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-ffa56ad0-e7c7-4371-8fbf-6f8f2a0efbc5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:40.436902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:40.437813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:40.438723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:40.438723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:40.438723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:40.438723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:40.438723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:40.438723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:40.438723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:40.438723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:40.438723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 18:37:40.438723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:40.438723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 259/506] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:40.439297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bde930-f184-449d-b75d-43f2c9730f02 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:40.439411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bde930-f184-449d-b75d-43f2c9730f02 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:40.439689 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bde930-f184-449d-b75d-43f2c9730f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:40.458434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-43bde930-f184-449d-b75d-43f2c9730f02 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:40.459633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-43bde930-f184-449d-b75d-43f2c9730f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:40.488035 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00c5fd-5bf3-4a28-925b-f7032ddb246a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:40.500822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:40.523916 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d188df-abb2-4a1c-9e26-53667eeb8da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:40.524805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d188df-abb2-4a1c-9e26-53667eeb8da8 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:40.525175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d188df-abb2-4a1c-9e26-53667eeb8da8 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:40.525460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d188df-abb2-4a1c-9e26-53667eeb8da8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:40.528921 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 260/507] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:40.529752 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bde930-f184-449d-b75d-43f2c9730f02 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:40.590698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:40.597771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d188df-abb2-4a1c-9e26-53667eeb8da8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:40.599240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d188df-abb2-4a1c-9e26-53667eeb8da8 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:40.642790 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bde930-f184-449d-b75d-43f2c9730f02 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:40.677363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff727c45-73f9-40cd-b016-d7a40ac47a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:40.678002 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff727c45-73f9-40cd-b016-d7a40ac47a1c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:40.678280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff727c45-73f9-40cd-b016-d7a40ac47a1c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:40.678552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff727c45-73f9-40cd-b016-d7a40ac47a1c None admin] PATH_INFO: `/v3/users/glance` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:40.686897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff727c45-73f9-40cd-b016-d7a40ac47a1c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:40.688480 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff727c45-73f9-40cd-b016-d7a40ac47a1c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-ff727c45-73f9-40cd-b016-d7a40ac47a1c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:40.700194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:40.701150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:40.702532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:40.702532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:40.702532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:40.702532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:40.702532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:40.702532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:40.702532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:40.702532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:40.702532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 03 18:37:40.702532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:40.703052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 248/508] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:40.703052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d188df-abb2-4a1c-9e26-53667eeb8da8 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:40.703174 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 261/509] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:40.703174 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff727c45-73f9-40cd-b016-d7a40ac47a1c None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:40.744246 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:40.801996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:40.809069 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d188df-abb2-4a1c-9e26-53667eeb8da8 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:40.812754 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff727c45-73f9-40cd-b016-d7a40ac47a1c None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:40.846058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d5d54-8630-4d69-bfd5-c6234508ad96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:40.846694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d5d54-8630-4d69-bfd5-c6234508ad96 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:40.846798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b1533-d4ab-448d-ade2-2277d3519bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:40.846990 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d5d54-8630-4d69-bfd5-c6234508ad96 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:40.847267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d5d54-8630-4d69-bfd5-c6234508ad96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:40.847375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b1533-d4ab-448d-ade2-2277d3519bb4 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:40.847622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b1533-d4ab-448d-ade2-2277d3519bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:40.847900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b1533-d4ab-448d-ade2-2277d3519bb4 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:40.848743 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b1533-d4ab-448d-ade2-2277d3519bb4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:40.849058 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b1533-d4ab-448d-ade2-2277d3519bb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:40.850497 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b1533-d4ab-448d-ade2-2277d3519bb4 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:40.858037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d5d54-8630-4d69-bfd5-c6234508ad96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:40.859462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d5d54-8630-4d69-bfd5-c6234508ad96 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:40.874417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-d06d5d54-8630-4d69-bfd5-c6234508ad96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:40.874417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:40.874417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:40.874417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:40.874417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:40.874417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:40.874417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:40.874417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:40.874417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:40.874417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:40.874417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:40.874417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:40.874417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:40.874417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:40.874417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:40.875311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:40.876167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:40.876167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:40.876167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:40.876167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:40.876167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 249/510] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:40.876480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d5d54-8630-4d69-bfd5-c6234508ad96 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:40.889739 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 262/511] 10.222.0.69 () {56 vars in 1137 bytes} [Wed Aug 3 18:37:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:40.890588 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b1533-d4ab-448d-ade2-2277d3519bb4 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:40.981676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d5d54-8630-4d69-bfd5-c6234508ad96 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:40.985596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:40.995841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b1533-d4ab-448d-ade2-2277d3519bb4 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:41.016951 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a161c40-5c0a-41e3-9a85-61164a139354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:41.017629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a161c40-5c0a-41e3-9a85-61164a139354 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:41.017914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a161c40-5c0a-41e3-9a85-61164a139354 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:41.018188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a161c40-5c0a-41e3-9a85-61164a139354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:41.029328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a161c40-5c0a-41e3-9a85-61164a139354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:41.030267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5338080-331a-476c-92f0-d4163fa5befd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:41.030815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a161c40-5c0a-41e3-9a85-61164a139354 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:41.030942 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5338080-331a-476c-92f0-d4163fa5befd None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:41.031191 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5338080-331a-476c-92f0-d4163fa5befd None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:41.031462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5338080-331a-476c-92f0-d4163fa5befd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:41.038704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:41.042806 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5338080-331a-476c-92f0-d4163fa5befd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:41.044274 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5338080-331a-476c-92f0-d4163fa5befd None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:41.046311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-9a161c40-5c0a-41e3-9a85-61164a139354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:41.046311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:41.046311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:41.046311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:41.046311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:41.046311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:41.046311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:41.046311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:41.046311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:41.046311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:41.046311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:41.046311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:41.046311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:41.046311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:41.046311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:41.047312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:41.048426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:41.048426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:41.048426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:41.048426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:41.050355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 250/512] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:41.051139 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a161c40-5c0a-41e3-9a85-61164a139354 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:41.063403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-b5338080-331a-476c-92f0-d4163fa5befd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:41.063403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:41.063403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:41.063403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:41.063403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:41.063403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:41.063403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:41.063403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:41.063403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:41.063403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:41.063403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:41.063403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:41.063403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:41.063403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:41.063403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:41.064324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:41.065194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:41.065194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:41.065194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:41.065194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:41.065194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 263/513] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:41.065780 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5338080-331a-476c-92f0-d4163fa5befd None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:41.157400 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a161c40-5c0a-41e3-9a85-61164a139354 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:41.167047 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5338080-331a-476c-92f0-d4163fa5befd None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:41.190928 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab7176-c34c-48c0-ad95-9e88678f5291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:41.191543 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab7176-c34c-48c0-ad95-9e88678f5291 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:41.191828 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab7176-c34c-48c0-ad95-9e88678f5291 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:41.192109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab7176-c34c-48c0-ad95-9e88678f5291 None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:41.192941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab7176-c34c-48c0-ad95-9e88678f5291 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:41.193259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab7176-c34c-48c0-ad95-9e88678f5291 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:41.194703 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab7176-c34c-48c0-ad95-9e88678f5291 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:41.199476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0597e-4a69-4f93-8932-b8bfbda9bd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:41.200090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0597e-4a69-4f93-8932-b8bfbda9bd7f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:41.200396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0597e-4a69-4f93-8932-b8bfbda9bd7f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:41.200680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0597e-4a69-4f93-8932-b8bfbda9bd7f None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:41.201373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0597e-4a69-4f93-8932-b8bfbda9bd7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:41.201675 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0597e-4a69-4f93-8932-b8bfbda9bd7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:41.203043 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0597e-4a69-4f93-8932-b8bfbda9bd7f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:41.212707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 251/514] 10.222.0.69 () {56 vars in 1137 bytes} [Wed Aug 3 18:37:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:41.213641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab7176-c34c-48c0-ad95-9e88678f5291 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:41.218016 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 264/515] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:41.218016 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0597e-4a69-4f93-8932-b8bfbda9bd7f None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:41.220731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:41.274793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:41.317662 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0597e-4a69-4f93-8932-b8bfbda9bd7f None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:41.318629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab7176-c34c-48c0-ad95-9e88678f5291 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:41.353799 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd263359-c5b5-413f-9fbd-afa9238977bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:41.354218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0dc16-9666-48a2-91ca-f80169b1b3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:41.354416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd263359-c5b5-413f-9fbd-afa9238977bf None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:41.354700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd263359-c5b5-413f-9fbd-afa9238977bf None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:41.354833 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0dc16-9666-48a2-91ca-f80169b1b3af None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:41.354978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd263359-c5b5-413f-9fbd-afa9238977bf None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:41.355115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0dc16-9666-48a2-91ca-f80169b1b3af None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:41.355389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0dc16-9666-48a2-91ca-f80169b1b3af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:41.355671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd263359-c5b5-413f-9fbd-afa9238977bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:41.355979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd263359-c5b5-413f-9fbd-afa9238977bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:41.357399 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd263359-c5b5-413f-9fbd-afa9238977bf None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:41.368679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 265/516] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:41.369466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd263359-c5b5-413f-9fbd-afa9238977bf None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:41.424853 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0dc16-9666-48a2-91ca-f80169b1b3af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:41.426314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0dc16-9666-48a2-91ca-f80169b1b3af None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:41.432015 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:41.472173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd263359-c5b5-413f-9fbd-afa9238977bf None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:41.484869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:41.507724 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa64ba-4743-4800-be55-7305375ea83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:41.508900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa64ba-4743-4800-be55-7305375ea83e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:41.509464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa64ba-4743-4800-be55-7305375ea83e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:41.509464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa64ba-4743-4800-be55-7305375ea83e None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:41.510118 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa64ba-4743-4800-be55-7305375ea83e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:41.510429 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa64ba-4743-4800-be55-7305375ea83e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:41.511804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa64ba-4743-4800-be55-7305375ea83e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:41.523385 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 266/517] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:41.525260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa64ba-4743-4800-be55-7305375ea83e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:41.537179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 252/518] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:41.537179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0dc16-9666-48a2-91ca-f80169b1b3af None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:41.630173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa64ba-4743-4800-be55-7305375ea83e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:41.634349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0dc16-9666-48a2-91ca-f80169b1b3af None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:41.654161 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:41.665211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e53ca-fbb8-4019-b481-b08b5a1d661f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:41.665828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e53ca-fbb8-4019-b481-b08b5a1d661f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:41.666113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e53ca-fbb8-4019-b481-b08b5a1d661f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:41.666388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e53ca-fbb8-4019-b481-b08b5a1d661f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:41.668796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb021eb-e687-46f2-8a55-ecf4969f5c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:41.669510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb021eb-e687-46f2-8a55-ecf4969f5c48 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:41.669740 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb021eb-e687-46f2-8a55-ecf4969f5c48 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:41.669989 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb021eb-e687-46f2-8a55-ecf4969f5c48 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:41.676994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb021eb-e687-46f2-8a55-ecf4969f5c48 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:41.678433 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb021eb-e687-46f2-8a55-ecf4969f5c48 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-2cb021eb-e687-46f2-8a55-ecf4969f5c48 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:41.693791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:41.694871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:41.696343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:41.696343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:41.696343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:41.696343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:41.696343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:41.696343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:41.696343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:41.696343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:41.696343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 18:37:41.696343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:41.696343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 253/519] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:41.696343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb021eb-e687-46f2-8a55-ecf4969f5c48 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:41.718794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:41.745770 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e53ca-fbb8-4019-b481-b08b5a1d661f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:41.747249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e53ca-fbb8-4019-b481-b08b5a1d661f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:41.804677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb021eb-e687-46f2-8a55-ecf4969f5c48 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:41.838773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55f08e-a63c-48fb-9c67-988789355f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:41.839386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55f08e-a63c-48fb-9c67-988789355f1d None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:41.839670 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55f08e-a63c-48fb-9c67-988789355f1d None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:41.839944 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55f08e-a63c-48fb-9c67-988789355f1d None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:41.840669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55f08e-a63c-48fb-9c67-988789355f1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:41.842096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55f08e-a63c-48fb-9c67-988789355f1d None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:41.853183 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 267/520] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:41.854103 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e53ca-fbb8-4019-b481-b08b5a1d661f None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:41.866207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 254/521] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:37:41 2022] POST /identity/v3/users => generated 247 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:37:41.867032 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55f08e-a63c-48fb-9c67-988789355f1d None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:41.883886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:41.968748 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:41.991901 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e53ca-fbb8-4019-b481-b08b5a1d661f None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:42.001206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55f08e-a63c-48fb-9c67-988789355f1d None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:42.035821 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a248d3-abe6-4c35-a2a1-28a4c3c68807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:42.036026 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a248d3-abe6-4c35-a2a1-28a4c3c68807 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:42.036281 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a248d3-abe6-4c35-a2a1-28a4c3c68807 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:42.036576 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a248d3-abe6-4c35-a2a1-28a4c3c68807 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:42.044667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a248d3-abe6-4c35-a2a1-28a4c3c68807 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:42.046429 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a248d3-abe6-4c35-a2a1-28a4c3c68807 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:42.049947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185a15e-1b71-4c44-82d4-ea80eb50e8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:42.050601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185a15e-1b71-4c44-82d4-ea80eb50e8e2 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:42.050882 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185a15e-1b71-4c44-82d4-ea80eb50e8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:42.051156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185a15e-1b71-4c44-82d4-ea80eb50e8e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:42.060329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-a3a248d3-abe6-4c35-a2a1-28a4c3c68807 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:42.060329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:42.060329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:42.060329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:42.060329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:42.060329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:42.060329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:42.060329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:42.060329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:42.060329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:42.060329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:42.060329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:42.060329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:37:42.060329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:37:42.060329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:42.061189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:42.062037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:37:42.062037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:37:42.062037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:37:42.062037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:37:42.062037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:37:42.062037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:37:42.062037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:42.062037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:42.062037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 268/522] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:42.063005 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a248d3-abe6-4c35-a2a1-28a4c3c68807 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:42.068824 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185a15e-1b71-4c44-82d4-ea80eb50e8e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:42.070291 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185a15e-1b71-4c44-82d4-ea80eb50e8e2 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:42.088423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-b185a15e-1b71-4c44-82d4-ea80eb50e8e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:42.088423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:42.088423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:42.088423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:42.088423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:42.088423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:42.088423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:42.088423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:42.088423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:42.088423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:42.088423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:42.088423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:42.088423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:42.088423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:42.088423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:42.089516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:42.090404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:42.090404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:42.090404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:42.090404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:42.090404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 255/523] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:42.091476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185a15e-1b71-4c44-82d4-ea80eb50e8e2 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:42.170354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:42.189169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a248d3-abe6-4c35-a2a1-28a4c3c68807 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:42.215301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185a15e-1b71-4c44-82d4-ea80eb50e8e2 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:42.226240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202adab-f238-4dd3-be8b-93702f8597ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:42.226862 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202adab-f238-4dd3-be8b-93702f8597ab None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:42.227259 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202adab-f238-4dd3-be8b-93702f8597ab None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:42.227597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202adab-f238-4dd3-be8b-93702f8597ab None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:42.228444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202adab-f238-4dd3-be8b-93702f8597ab None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:42.228759 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202adab-f238-4dd3-be8b-93702f8597ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:42.230199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202adab-f238-4dd3-be8b-93702f8597ab None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:42.243761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:42.249620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 269/524] 10.222.0.69 () {56 vars in 1139 bytes} [Wed Aug 3 18:37:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:42.254584 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202adab-f238-4dd3-be8b-93702f8597ab None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:42.259728 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c6def-42d3-4971-91d3-27c28ef56fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:42.260733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c6def-42d3-4971-91d3-27c28ef56fdd None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:42.260844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c6def-42d3-4971-91d3-27c28ef56fdd None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:42.261196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c6def-42d3-4971-91d3-27c28ef56fdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:42.273106 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c6def-42d3-4971-91d3-27c28ef56fdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:42.274981 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c6def-42d3-4971-91d3-27c28ef56fdd None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:42.291977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-0b0c6def-42d3-4971-91d3-27c28ef56fdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:42.291977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:42.291977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:42.291977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:42.291977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:42.291977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:42.291977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:42.291977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:42.291977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:42.291977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:42.291977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:42.291977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:42.291977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:42.291977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:42.291977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:42.292935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:42.298676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:42.298676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:42.298676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:42.298676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:42.298676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 256/525] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:42.298676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c6def-42d3-4971-91d3-27c28ef56fdd None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:42.372546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202adab-f238-4dd3-be8b-93702f8597ab None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:42.402907 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c6def-42d3-4971-91d3-27c28ef56fdd None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:42.411368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83e14c-7662-44cd-aa17-b879f18d47d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:42.411995 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83e14c-7662-44cd-aa17-b879f18d47d4 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:42.412277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83e14c-7662-44cd-aa17-b879f18d47d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:42.412579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83e14c-7662-44cd-aa17-b879f18d47d4 None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:42.413336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83e14c-7662-44cd-aa17-b879f18d47d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:42.413646 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83e14c-7662-44cd-aa17-b879f18d47d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:42.415629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83e14c-7662-44cd-aa17-b879f18d47d4 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:42.436910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 270/526] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:37:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:42.438607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83e14c-7662-44cd-aa17-b879f18d47d4 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:42.441271 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6f948-0ee6-4ecc-a737-eafbc5e2b28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:42.441892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6f948-0ee6-4ecc-a737-eafbc5e2b28f None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:42.442194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6f948-0ee6-4ecc-a737-eafbc5e2b28f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:42.442477 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:42.442619 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6f948-0ee6-4ecc-a737-eafbc5e2b28f None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:42.443300 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6f948-0ee6-4ecc-a737-eafbc5e2b28f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:42.443611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6f948-0ee6-4ecc-a737-eafbc5e2b28f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:42.446048 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6f948-0ee6-4ecc-a737-eafbc5e2b28f None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:42.459887 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 257/527] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:42.460745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6f948-0ee6-4ecc-a737-eafbc5e2b28f None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:42.506487 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:42.559007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83e14c-7662-44cd-aa17-b879f18d47d4 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:42.575016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6f948-0ee6-4ecc-a737-eafbc5e2b28f None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:42.597523 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5bb03-77ad-4d46-98c5-b5ae22cf2dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:42.598306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5bb03-77ad-4d46-98c5-b5ae22cf2dea None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:42.598587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5bb03-77ad-4d46-98c5-b5ae22cf2dea None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:42.598859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5bb03-77ad-4d46-98c5-b5ae22cf2dea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:42.612276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fb228-2cf4-44a4-bdbd-eb4a2638c531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:42.612933 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fb228-2cf4-44a4-bdbd-eb4a2638c531 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:42.613329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fb228-2cf4-44a4-bdbd-eb4a2638c531 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:42.613601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fb228-2cf4-44a4-bdbd-eb4a2638c531 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:42.614302 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fb228-2cf4-44a4-bdbd-eb4a2638c531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:42.614608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fb228-2cf4-44a4-bdbd-eb4a2638c531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:42.615974 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fb228-2cf4-44a4-bdbd-eb4a2638c531 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:42.634159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 258/528] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:42.634159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fb228-2cf4-44a4-bdbd-eb4a2638c531 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:42.681272 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5bb03-77ad-4d46-98c5-b5ae22cf2dea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:42.682744 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5bb03-77ad-4d46-98c5-b5ae22cf2dea None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:42.684897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:42.735663 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:42.740480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fb228-2cf4-44a4-bdbd-eb4a2638c531 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:42.777011 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340529f-59f4-456f-b20a-b60342292b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:42.777629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340529f-59f4-456f-b20a-b60342292b03 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:42.777914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340529f-59f4-456f-b20a-b60342292b03 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:42.778192 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340529f-59f4-456f-b20a-b60342292b03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:42.778990 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340529f-59f4-456f-b20a-b60342292b03 None admin] RBAC: Adding query filter params (role.id=d67cdb8bae3648fd8f3589ab33176d14, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=850a50cfe4394febbafb0908b08888b2) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:42.779299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340529f-59f4-456f-b20a-b60342292b03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:42.780973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340529f-59f4-456f-b20a-b60342292b03 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:42.796319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 259/529] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:42 2022] GET /identity/v3/role_assignments?user.id=850a50cfe4394febbafb0908b08888b2&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=d67cdb8bae3648fd8f3589ab33176d14 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:42.796319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340529f-59f4-456f-b20a-b60342292b03 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:42.804919 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 271/530] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:42.806659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5bb03-77ad-4d46-98c5-b5ae22cf2dea None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:42.922770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340529f-59f4-456f-b20a-b60342292b03 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:42.942682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5bb03-77ad-4d46-98c5-b5ae22cf2dea None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:42.944863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:42.970351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e073af0-7c79-49ce-911b-25d6bb4fa180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:42.970980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e073af0-7c79-49ce-911b-25d6bb4fa180 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:42.971261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e073af0-7c79-49ce-911b-25d6bb4fa180 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:42.971541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e073af0-7c79-49ce-911b-25d6bb4fa180 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:42.979883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e073af0-7c79-49ce-911b-25d6bb4fa180 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:42.981857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e073af0-7c79-49ce-911b-25d6bb4fa180 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:42.986921 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3f3b7-96a6-4b4c-83b0-abc6dc9f23d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:42.987913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3f3b7-96a6-4b4c-83b0-abc6dc9f23d6 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:42.988497 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3f3b7-96a6-4b4c-83b0-abc6dc9f23d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:42.988970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3f3b7-96a6-4b4c-83b0-abc6dc9f23d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:42.995286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-7e073af0-7c79-49ce-911b-25d6bb4fa180 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:42.995286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:42.995286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:42.995286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:42.995286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:42.995286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:42.995286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:42.995286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:42.995286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:42.995286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:42.995286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:42.995286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:42.995286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:37:42.995286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:37:42.995286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:42.996174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:42.997076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:37:42.997076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:37:42.997076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:37:42.997076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:37:42.997076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:37:42.997076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:37:42.997076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:42.997076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:42.999493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3f3b7-96a6-4b4c-83b0-abc6dc9f23d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:43.001165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3f3b7-96a6-4b4c-83b0-abc6dc9f23d6 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:43.001493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 260/531] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:43.002421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e073af0-7c79-49ce-911b-25d6bb4fa180 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:43.014943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-3bc3f3b7-96a6-4b4c-83b0-abc6dc9f23d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:43.014943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:43.014943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:43.014943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:43.014943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:43.014943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:43.014943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:43.014943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:43.014943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:43.014943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:43.014943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:43.014943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:43.014943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:37:43.014943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:37:43.014943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:43.015884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:43.016825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:37:43.016825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:37:43.016825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:37:43.016825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:37:43.016825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:37:43.016825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:37:43.016825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:43.016825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:43.025031 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 272/532] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:43.025031 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3f3b7-96a6-4b4c-83b0-abc6dc9f23d6 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:43.042575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:43.129914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e073af0-7c79-49ce-911b-25d6bb4fa180 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:43.147861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3f3b7-96a6-4b4c-83b0-abc6dc9f23d6 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:43.168794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d4bfa-1040-40a8-9038-54862a7022d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:43.169595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d4bfa-1040-40a8-9038-54862a7022d6 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:43.170517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d4bfa-1040-40a8-9038-54862a7022d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:43.170873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d4bfa-1040-40a8-9038-54862a7022d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:43.186894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d4bfa-1040-40a8-9038-54862a7022d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:43.188626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d4bfa-1040-40a8-9038-54862a7022d6 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:43.189317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf52f1f-0dbd-4286-a7ef-ee78c69ef229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:43.190017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf52f1f-0dbd-4286-a7ef-ee78c69ef229 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:43.190358 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf52f1f-0dbd-4286-a7ef-ee78c69ef229 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:43.190640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf52f1f-0dbd-4286-a7ef-ee78c69ef229 None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:43.191664 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf52f1f-0dbd-4286-a7ef-ee78c69ef229 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:43.191980 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf52f1f-0dbd-4286-a7ef-ee78c69ef229 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:43.193911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf52f1f-0dbd-4286-a7ef-ee78c69ef229 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:43.205891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-bf8d4bfa-1040-40a8-9038-54862a7022d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:43.205891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:43.205891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:43.205891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:43.205891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:43.205891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:43.205891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:43.205891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:43.205891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:43.205891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:43.205891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:43.205891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:43.205891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:43.205891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:43.205891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:43.206789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:43.207646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:43.207646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:43.207646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:43.207646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:43.209806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 261/533] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:43.210902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d4bfa-1040-40a8-9038-54862a7022d6 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:43.213192 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 273/534] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:37:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:43.214369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf52f1f-0dbd-4286-a7ef-ee78c69ef229 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:43.248644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:43.318215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:43.346572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf52f1f-0dbd-4286-a7ef-ee78c69ef229 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:43.353842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d4bfa-1040-40a8-9038-54862a7022d6 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:43.385337 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae77dc-188b-4b0d-8da1-acb0690e8b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:43.386067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae77dc-188b-4b0d-8da1-acb0690e8b84 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:43.386355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae77dc-188b-4b0d-8da1-acb0690e8b84 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:43.386631 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae77dc-188b-4b0d-8da1-acb0690e8b84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:43.387443 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae77dc-188b-4b0d-8da1-acb0690e8b84 None admin] RBAC: Adding query filter params (role.id=d67cdb8bae3648fd8f3589ab33176d14, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=c7d0fde945184c93b8fd91d15b66f534) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:43.387757 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae77dc-188b-4b0d-8da1-acb0690e8b84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:43.389599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae77dc-188b-4b0d-8da1-acb0690e8b84 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:43.396376 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 274/535] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:43 2022] GET /identity/v3/role_assignments?user.id=c7d0fde945184c93b8fd91d15b66f534&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=d67cdb8bae3648fd8f3589ab33176d14 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:43.397292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae77dc-188b-4b0d-8da1-acb0690e8b84 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:43.399417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be24657-4506-467c-afef-b7bec6a49bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:43.400060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be24657-4506-467c-afef-b7bec6a49bb5 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:43.400649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be24657-4506-467c-afef-b7bec6a49bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:43.400938 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be24657-4506-467c-afef-b7bec6a49bb5 None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:43.402993 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be24657-4506-467c-afef-b7bec6a49bb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:43.403311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be24657-4506-467c-afef-b7bec6a49bb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:43.404836 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be24657-4506-467c-afef-b7bec6a49bb5 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:43.435217 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 262/536] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:37:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:43.442298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be24657-4506-467c-afef-b7bec6a49bb5 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:43.566014 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:43.609886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be24657-4506-467c-afef-b7bec6a49bb5 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:43.610240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae77dc-188b-4b0d-8da1-acb0690e8b84 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:43.659617 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440eafe-4a2f-465d-9de5-87c3d1ebaef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:43.660247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440eafe-4a2f-465d-9de5-87c3d1ebaef2 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:43.660558 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440eafe-4a2f-465d-9de5-87c3d1ebaef2 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:43.660837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440eafe-4a2f-465d-9de5-87c3d1ebaef2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:43.661644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440eafe-4a2f-465d-9de5-87c3d1ebaef2 None admin] RBAC: Adding query filter params (role.id=d67cdb8bae3648fd8f3589ab33176d14, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=4799b0c9e5aa47d389121d2183666d3d) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:43.661957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440eafe-4a2f-465d-9de5-87c3d1ebaef2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:43.663392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440eafe-4a2f-465d-9de5-87c3d1ebaef2 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:43.666507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:43.673866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 263/537] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:43 2022] GET /identity/v3/role_assignments?user.id=4799b0c9e5aa47d389121d2183666d3d&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=d67cdb8bae3648fd8f3589ab33176d14 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:43.677091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804caf8-d89e-4c92-ab09-f3399d06905b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:43.677967 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804caf8-d89e-4c92-ab09-f3399d06905b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:43.678506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804caf8-d89e-4c92-ab09-f3399d06905b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:43.679009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804caf8-d89e-4c92-ab09-f3399d06905b None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:43.679934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804caf8-d89e-4c92-ab09-f3399d06905b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:43.680603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804caf8-d89e-4c92-ab09-f3399d06905b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:43.682460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804caf8-d89e-4c92-ab09-f3399d06905b None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:43.697734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 275/538] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:43.704378 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440eafe-4a2f-465d-9de5-87c3d1ebaef2 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:43.848022 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440eafe-4a2f-465d-9de5-87c3d1ebaef2 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:43.895664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a9569-1929-4880-8002-1be50641c375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:43.896359 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a9569-1929-4880-8002-1be50641c375 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:43.896651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a9569-1929-4880-8002-1be50641c375 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:43.896930 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a9569-1929-4880-8002-1be50641c375 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:43.905675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a9569-1929-4880-8002-1be50641c375 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:43.907146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a9569-1929-4880-8002-1be50641c375 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:43.930457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-162a9569-1929-4880-8002-1be50641c375 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:43.930457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:43.930457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:43.930457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:43.930457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:43.930457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:43.930457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:43.930457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:43.930457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:43.930457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:43.930457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:43.930457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:43.930457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:37:43.930457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:37:43.930457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:43.931370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:43.932204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:37:43.932204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:37:43.932204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:37:43.932204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:37:43.932204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:37:43.932204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:37:43.932204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:43.932204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:43.940378 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 264/539] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:43.942438 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804caf8-d89e-4c92-ab09-f3399d06905b None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:44.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfced1-72a4-403a-b964-73357de6a780 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:44.006833 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfced1-72a4-403a-b964-73357de6a780 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:44.007069 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfced1-72a4-403a-b964-73357de6a780 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:44.009310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 265/540] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:37:44.014894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33130dec-4535-4279-a9b2-6ecf87387947 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:44.015077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33130dec-4535-4279-a9b2-6ecf87387947 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:44.015285 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33130dec-4535-4279-a9b2-6ecf87387947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:44.041181 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-33130dec-4535-4279-a9b2-6ecf87387947 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:44.042414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-33130dec-4535-4279-a9b2-6ecf87387947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:44.080159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804caf8-d89e-4c92-ab09-f3399d06905b None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:44.104755 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:44.120815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 266/541] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:44.121163 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba43fed-8490-4096-bf97-4ea0e3bacf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:44.121977 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba43fed-8490-4096-bf97-4ea0e3bacf79 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:44.122447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba43fed-8490-4096-bf97-4ea0e3bacf79 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:44.122957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba43fed-8490-4096-bf97-4ea0e3bacf79 None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:44.123881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba43fed-8490-4096-bf97-4ea0e3bacf79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:44.124403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba43fed-8490-4096-bf97-4ea0e3bacf79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:44.126009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba43fed-8490-4096-bf97-4ea0e3bacf79 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:44.139981 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 276/542] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:37:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:44.145989 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33130dec-4535-4279-a9b2-6ecf87387947 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:44.147120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba43fed-8490-4096-bf97-4ea0e3bacf79 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:44.160661 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:44.256183 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33130dec-4535-4279-a9b2-6ecf87387947 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:44.293230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9941f8-2ed3-446f-add2-dc56bd9035b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:44.293917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9941f8-2ed3-446f-add2-dc56bd9035b5 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:44.294266 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9941f8-2ed3-446f-add2-dc56bd9035b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:44.294635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9941f8-2ed3-446f-add2-dc56bd9035b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:44.314655 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9941f8-2ed3-446f-add2-dc56bd9035b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-8d9941f8-2ed3-446f-add2-dc56bd9035b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:44.323999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:44.324949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:44.325770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:44.325770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:44.325770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:44.325770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:44.325770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:44.325770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:44.325770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:44.325770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:44.325770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:44.325770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:44.325770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:44.325770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:44.332543 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 267/543] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:44.336831 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9941f8-2ed3-446f-add2-dc56bd9035b5 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:44.349807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba43fed-8490-4096-bf97-4ea0e3bacf79 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:44.400873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:44.438772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77316a-441e-40ae-b848-726fc32a5c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:44.439408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77316a-441e-40ae-b848-726fc32a5c2c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:44.439693 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77316a-441e-40ae-b848-726fc32a5c2c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:44.439973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77316a-441e-40ae-b848-726fc32a5c2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:44.440816 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77316a-441e-40ae-b848-726fc32a5c2c None admin] RBAC: Adding query filter params (role.id=d67cdb8bae3648fd8f3589ab33176d14, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=0eb9ad672e5444098d71b7c9a27b72e8) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:44.441233 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77316a-441e-40ae-b848-726fc32a5c2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:44.442806 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77316a-441e-40ae-b848-726fc32a5c2c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:44.448997 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9941f8-2ed3-446f-add2-dc56bd9035b5 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:44.450068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 277/544] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:44 2022] GET /identity/v3/role_assignments?user.id=0eb9ad672e5444098d71b7c9a27b72e8&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=d67cdb8bae3648fd8f3589ab33176d14 => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:44.490805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a02dc-339f-4e79-8dc1-95a3371cca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:44.491444 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a02dc-339f-4e79-8dc1-95a3371cca81 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:44.491730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a02dc-339f-4e79-8dc1-95a3371cca81 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:44.492007 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a02dc-339f-4e79-8dc1-95a3371cca81 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:44.493339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a02dc-339f-4e79-8dc1-95a3371cca81 None admin] RBAC: Adding query filter params (name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:44.493648 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a02dc-339f-4e79-8dc1-95a3371cca81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:44.495211 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a02dc-339f-4e79-8dc1-95a3371cca81 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:44.505336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 268/545] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:37:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:44.512578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77316a-441e-40ae-b848-726fc32a5c2c None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:44.594333 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:44.631567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77316a-441e-40ae-b848-726fc32a5c2c None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:44.669051 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79bd67-df70-478f-8754-3b7c3b4ffe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:44.670099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79bd67-df70-478f-8754-3b7c3b4ffe3c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:44.670580 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79bd67-df70-478f-8754-3b7c3b4ffe3c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:44.671120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79bd67-df70-478f-8754-3b7c3b4ffe3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:44.683609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79bd67-df70-478f-8754-3b7c3b4ffe3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:44.685409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79bd67-df70-478f-8754-3b7c3b4ffe3c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:44.701704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-ac79bd67-df70-478f-8754-3b7c3b4ffe3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:44.701704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:44.701704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:44.701704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:44.701704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:44.701704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:44.701704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:44.701704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:44.701704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:44.701704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:44.701704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:44.701704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:44.701704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:44.701704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:44.701704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:44.702602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:44.703459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:44.703459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:44.703459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:44.703459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:44.705683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 278/546] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:44.711083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a02dc-339f-4e79-8dc1-95a3371cca81 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:44.815734 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a02dc-339f-4e79-8dc1-95a3371cca81 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:44.827545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:44.852758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ba8a4-450e-4c16-8db2-162ea8a95a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:44.853313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ba8a4-450e-4c16-8db2-162ea8a95a2c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:44.853598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ba8a4-450e-4c16-8db2-162ea8a95a2c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:44.853876 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ba8a4-450e-4c16-8db2-162ea8a95a2c None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:44.854583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ba8a4-450e-4c16-8db2-162ea8a95a2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:44.854888 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ba8a4-450e-4c16-8db2-162ea8a95a2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:44.856265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ba8a4-450e-4c16-8db2-162ea8a95a2c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:44.869512 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 269/547] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:44.876644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79bd67-df70-478f-8754-3b7c3b4ffe3c None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:45.046052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79bd67-df70-478f-8754-3b7c3b4ffe3c None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:45.076614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:45.094519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d491a-3aee-4f8e-9f4a-168af2a3ecec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:45.095213 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d491a-3aee-4f8e-9f4a-168af2a3ecec None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:45.095497 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d491a-3aee-4f8e-9f4a-168af2a3ecec None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:45.095809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d491a-3aee-4f8e-9f4a-168af2a3ecec None admin] PATH_INFO: `/v3/users/swift` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:45.109071 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d491a-3aee-4f8e-9f4a-168af2a3ecec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:45.110516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d491a-3aee-4f8e-9f4a-168af2a3ecec None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-ca5d491a-3aee-4f8e-9f4a-168af2a3ecec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:45.125771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:45.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:45.129742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:45.129742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:45.129742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:45.129742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:45.129742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:45.129742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:45.129742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:45.129742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:45.129742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 18:37:45.129742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:45.129742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 279/548] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:37:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:45.134141 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ba8a4-450e-4c16-8db2-162ea8a95a2c None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:45.237986 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f805a-e80d-4f85-b1d3-956250942d78 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:45.238329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f805a-e80d-4f85-b1d3-956250942d78 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:45.238543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f805a-e80d-4f85-b1d3-956250942d78 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:45.240144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 280/549] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:37:45.245911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba4d67-506c-46d6-9af8-40964fe0345a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:45.246193 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba4d67-506c-46d6-9af8-40964fe0345a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:45.246467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba4d67-506c-46d6-9af8-40964fe0345a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:45.265911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d6ba4d67-506c-46d6-9af8-40964fe0345a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:45.267096 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d6ba4d67-506c-46d6-9af8-40964fe0345a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:45.338709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 281/550] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:45.341103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ba8a4-450e-4c16-8db2-162ea8a95a2c None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:45.361302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba4d67-506c-46d6-9af8-40964fe0345a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:45.391571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc3933-fe66-44ea-a1a1-dcd68ca4ed3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:45.392207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc3933-fe66-44ea-a1a1-dcd68ca4ed3d None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:45.392524 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc3933-fe66-44ea-a1a1-dcd68ca4ed3d None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:45.392807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc3933-fe66-44ea-a1a1-dcd68ca4ed3d None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:45.393625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc3933-fe66-44ea-a1a1-dcd68ca4ed3d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:45.393996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc3933-fe66-44ea-a1a1-dcd68ca4ed3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:45.395612 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc3933-fe66-44ea-a1a1-dcd68ca4ed3d None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:45.416108 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 270/551] 10.222.0.69 () {56 vars in 1135 bytes} [Wed Aug 3 18:37:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:45.422024 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc3933-fe66-44ea-a1a1-dcd68ca4ed3d None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:45.452660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:45.532317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc3933-fe66-44ea-a1a1-dcd68ca4ed3d None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:45.570435 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba4d67-506c-46d6-9af8-40964fe0345a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:45.571817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6acf13-ffb6-46ea-99a4-54b21fbe243f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:45.572529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6acf13-ffb6-46ea-99a4-54b21fbe243f None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:45.572815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6acf13-ffb6-46ea-99a4-54b21fbe243f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:45.573094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6acf13-ffb6-46ea-99a4-54b21fbe243f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:45.603800 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:45.656181 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6acf13-ffb6-46ea-99a4-54b21fbe243f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:45.658282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6acf13-ffb6-46ea-99a4-54b21fbe243f None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:45.694235 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a383f15-dd6f-4574-af5b-c46040b71432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:45.695109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a383f15-dd6f-4574-af5b-c46040b71432 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:45.695490 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a383f15-dd6f-4574-af5b-c46040b71432 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:45.696061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a383f15-dd6f-4574-af5b-c46040b71432 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:45.708710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a383f15-dd6f-4574-af5b-c46040b71432 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:45.709697 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-3a383f15-dd6f-4574-af5b-c46040b71432 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:45.717409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:45.718464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:45.719911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:45.719911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:45.719911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:45.719911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:45.719911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:45.719911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:45.719911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:45.719911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:45.719911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:45.719911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:45.719911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:45.719911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:45.719911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 282/552] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:45.724420 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a383f15-dd6f-4574-af5b-c46040b71432 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:45.897071 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a383f15-dd6f-4574-af5b-c46040b71432 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:45.907516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 271/553] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:45.926833 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6acf13-ffb6-46ea-99a4-54b21fbe243f None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:45.949060 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592be9b-f863-4208-998d-2ebb94dc30db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:45.949785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592be9b-f863-4208-998d-2ebb94dc30db None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:45.950071 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592be9b-f863-4208-998d-2ebb94dc30db None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:45.950350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592be9b-f863-4208-998d-2ebb94dc30db None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:45.951117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592be9b-f863-4208-998d-2ebb94dc30db None admin] RBAC: Adding query filter params (name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:45.951428 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592be9b-f863-4208-998d-2ebb94dc30db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:45.953442 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592be9b-f863-4208-998d-2ebb94dc30db None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:45.963334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 283/554] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:37:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:45.968361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592be9b-f863-4208-998d-2ebb94dc30db None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:46.035110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:46.150436 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6acf13-ffb6-46ea-99a4-54b21fbe243f None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:46.173544 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592be9b-f863-4208-998d-2ebb94dc30db None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:46.197215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:46.215181 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83073603-2a63-4d48-91a6-3dd584f9b66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:46.216217 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83073603-2a63-4d48-91a6-3dd584f9b66c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:46.216734 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83073603-2a63-4d48-91a6-3dd584f9b66c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:46.217212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83073603-2a63-4d48-91a6-3dd584f9b66c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:46.226612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958474b1-fd86-47bf-b5e5-181311835805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:46.227978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958474b1-fd86-47bf-b5e5-181311835805 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:46.231206 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958474b1-fd86-47bf-b5e5-181311835805 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:46.231206 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958474b1-fd86-47bf-b5e5-181311835805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:46.233107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83073603-2a63-4d48-91a6-3dd584f9b66c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:46.234665 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83073603-2a63-4d48-91a6-3dd584f9b66c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:46.247157 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958474b1-fd86-47bf-b5e5-181311835805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:46.248961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958474b1-fd86-47bf-b5e5-181311835805 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:46.253918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-83073603-2a63-4d48-91a6-3dd584f9b66c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:46.253918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:46.253918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:46.253918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:46.253918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:46.253918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:46.253918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:46.253918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:46.253918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:46.253918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:46.253918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:46.253918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:46.253918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:46.253918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:46.253918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:46.254805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:46.255892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:46.255892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:46.255892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:46.255892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:46.255892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 272/555] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:46.257238 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62362dd7-1329-43f8-bf24-c7d6f9e93742 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:46.257586 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62362dd7-1329-43f8-bf24-c7d6f9e93742 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:46.257928 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62362dd7-1329-43f8-bf24-c7d6f9e93742 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:46.259977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 273/556] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:46.261789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8585126-b7d2-47e0-9767-eb79d2c45ca1 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:46.262140 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8585126-b7d2-47e0-9767-eb79d2c45ca1 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:46.262488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8585126-b7d2-47e0-9767-eb79d2c45ca1 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:46.264329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 274/557] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:46.265111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8585126-b7d2-47e0-9767-eb79d2c45ca1 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:46.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-958474b1-fd86-47bf-b5e5-181311835805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:46.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:46.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:46.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:46.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:46.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:46.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:46.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:46.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:46.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:46.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:46.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:46.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:46.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:46.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:46.272878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:46.274467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:46.274467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:46.274467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:46.274467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:46.274467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 284/558] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:46.274896 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed0293-15fe-4db6-8ace-a3eaa0a5dd5f None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:46.275186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed0293-15fe-4db6-8ace-a3eaa0a5dd5f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:46.279758 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed0293-15fe-4db6-8ace-a3eaa0a5dd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:46.294390 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-44ed0293-15fe-4db6-8ace-a3eaa0a5dd5f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:46.295588 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-44ed0293-15fe-4db6-8ace-a3eaa0a5dd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:46.374496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 285/559] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:46.377779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d7098-66ed-46be-b870-dadeb0981f39 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:46.378068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d7098-66ed-46be-b870-dadeb0981f39 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:46.378596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d7098-66ed-46be-b870-dadeb0981f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:46.385846 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8585126-b7d2-47e0-9767-eb79d2c45ca1 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:46.401940 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-df2d7098-66ed-46be-b870-dadeb0981f39 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:46.403137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-df2d7098-66ed-46be-b870-dadeb0981f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:46.414687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:46.425112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9dfba-b14b-45b1-8252-6dc776e6696b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:46.425662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9dfba-b14b-45b1-8252-6dc776e6696b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:46.425968 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9dfba-b14b-45b1-8252-6dc776e6696b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:46.426247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9dfba-b14b-45b1-8252-6dc776e6696b None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:46.426953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9dfba-b14b-45b1-8252-6dc776e6696b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:46.427259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9dfba-b14b-45b1-8252-6dc776e6696b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:46.428677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9dfba-b14b-45b1-8252-6dc776e6696b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:46.446977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 275/560] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:46.447510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9dfba-b14b-45b1-8252-6dc776e6696b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:46.477299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 286/561] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:46.478322 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d7098-66ed-46be-b870-dadeb0981f39 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:46.498427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:46.563780 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9dfba-b14b-45b1-8252-6dc776e6696b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:46.600075 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ddc74-4658-428d-9f97-4962f9461b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:46.600753 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ddc74-4658-428d-9f97-4962f9461b8a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:46.601033 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ddc74-4658-428d-9f97-4962f9461b8a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:46.601308 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ddc74-4658-428d-9f97-4962f9461b8a None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:46.601412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d7098-66ed-46be-b870-dadeb0981f39 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:46.602054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ddc74-4658-428d-9f97-4962f9461b8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:46.602627 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ddc74-4658-428d-9f97-4962f9461b8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:46.604180 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ddc74-4658-428d-9f97-4962f9461b8a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:46.616903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 276/562] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:46.618324 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ddc74-4658-428d-9f97-4962f9461b8a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:46.649040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102e583-dc06-4168-8da8-5b47aaeb2ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:46.650811 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102e583-dc06-4168-8da8-5b47aaeb2ab6 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:46.651094 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102e583-dc06-4168-8da8-5b47aaeb2ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:46.651415 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102e583-dc06-4168-8da8-5b47aaeb2ab6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:46.662803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102e583-dc06-4168-8da8-5b47aaeb2ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-9102e583-dc06-4168-8da8-5b47aaeb2ab6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:46.668252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:46.669334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:46.670178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:46.670178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:46.670178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:46.670178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:46.670178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:46.670178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:46.670178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:46.670178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:46.670178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:46.670178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:46.670178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:46.670178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:46.670178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 287/563] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:46.670943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102e583-dc06-4168-8da8-5b47aaeb2ab6 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:46.681398 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:46.730013 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:46.733505 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ddc74-4658-428d-9f97-4962f9461b8a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:46.770959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47feddc9-e5d1-4eb7-8f40-2a85b500eea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:46.771611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feddc9-e5d1-4eb7-8f40-2a85b500eea2 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:46.771890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feddc9-e5d1-4eb7-8f40-2a85b500eea2 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:46.772167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feddc9-e5d1-4eb7-8f40-2a85b500eea2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:46.779644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47feddc9-e5d1-4eb7-8f40-2a85b500eea2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:46.781111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47feddc9-e5d1-4eb7-8f40-2a85b500eea2 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:46.791510 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102e583-dc06-4168-8da8-5b47aaeb2ab6 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:46.793247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-47feddc9-e5d1-4eb7-8f40-2a85b500eea2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:46.793247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:46.793247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:46.793247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:46.793247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:46.793247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:46.793247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:46.793247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:46.793247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:46.793247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:46.793247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:46.793247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:46.793247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:37:46.793247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:37:46.793247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:46.794319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:46.795519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:37:46.795519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:37:46.795519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:37:46.795519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:37:46.795519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:37:46.795519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:37:46.795519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:46.795519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:46.799132 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 277/564] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:46.799964 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47feddc9-e5d1-4eb7-8f40-2a85b500eea2 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:46.832147 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c29189-9dfe-415f-8fd4-a2e7b526a8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:46.832820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c29189-9dfe-415f-8fd4-a2e7b526a8ae None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:46.833101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c29189-9dfe-415f-8fd4-a2e7b526a8ae None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:46.833374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c29189-9dfe-415f-8fd4-a2e7b526a8ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:46.844568 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c29189-9dfe-415f-8fd4-a2e7b526a8ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-68c29189-9dfe-415f-8fd4-a2e7b526a8ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:46.849424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:46.850535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:46.851677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:46.851677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:46.851677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:46.851677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:46.851677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:46.851677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:46.851677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:46.851677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:46.851677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:46.851677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:46.851677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:46.851677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:46.855759 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 288/565] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:46.856914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c29189-9dfe-415f-8fd4-a2e7b526a8ae None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:46.910897 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47feddc9-e5d1-4eb7-8f40-2a85b500eea2 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:46.924361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:46.947700 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa892c-434e-4f53-ad79-4b34033984e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:46.948369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa892c-434e-4f53-ad79-4b34033984e9 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:46.948656 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa892c-434e-4f53-ad79-4b34033984e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:46.948960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa892c-434e-4f53-ad79-4b34033984e9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:46.956529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa892c-434e-4f53-ad79-4b34033984e9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:46.957966 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa892c-434e-4f53-ad79-4b34033984e9 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:46.964144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c29189-9dfe-415f-8fd4-a2e7b526a8ae None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-7faa892c-434e-4f53-ad79-4b34033984e9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:46.970187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:46.971316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:46.972317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:46.972317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:46.972317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:46.972317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:46.972317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:46.972317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:46.972317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:46.972317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:46.972317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:37:46.972317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:46.972317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 278/566] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Aug 3 18:37:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:46.972939 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa892c-434e-4f53-ad79-4b34033984e9 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:46.996520 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:47.002200 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bb61e-c552-4329-aac5-9df190978d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:47.002842 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bb61e-c552-4329-aac5-9df190978d62 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:47.003125 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bb61e-c552-4329-aac5-9df190978d62 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:47.003658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bb61e-c552-4329-aac5-9df190978d62 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:47.004416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bb61e-c552-4329-aac5-9df190978d62 None admin] RBAC: Adding query filter params (name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:47.004729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bb61e-c552-4329-aac5-9df190978d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:47.006112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bb61e-c552-4329-aac5-9df190978d62 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:47.014982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 289/567] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:37:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:47.015790 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bb61e-c552-4329-aac5-9df190978d62 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:47.081224 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa892c-434e-4f53-ad79-4b34033984e9 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:47.115118 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51238d7c-355b-423b-bc5a-486d6ddf57cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:47.115857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51238d7c-355b-423b-bc5a-486d6ddf57cc None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:47.116199 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51238d7c-355b-423b-bc5a-486d6ddf57cc None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:47.116440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bb61e-c552-4329-aac5-9df190978d62 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:47.116562 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51238d7c-355b-423b-bc5a-486d6ddf57cc None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:47.117314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51238d7c-355b-423b-bc5a-486d6ddf57cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:47.117804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51238d7c-355b-423b-bc5a-486d6ddf57cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:47.119258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51238d7c-355b-423b-bc5a-486d6ddf57cc None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:47.131836 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 279/568] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:37:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:47.133817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c2ffa-894d-4685-828b-e901d5dafb97 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:47.134205 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c2ffa-894d-4685-828b-e901d5dafb97 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:47.134487 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c2ffa-894d-4685-828b-e901d5dafb97 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:47.136334 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 280/569] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:47.136485 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c2ffa-894d-4685-828b-e901d5dafb97 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:47.150948 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:47.160655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c93060-cc6b-4e3f-b5a6-db6f6a9859da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:47.161295 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c93060-cc6b-4e3f-b5a6-db6f6a9859da None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:47.161577 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c93060-cc6b-4e3f-b5a6-db6f6a9859da None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:47.161851 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c93060-cc6b-4e3f-b5a6-db6f6a9859da None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:47.162844 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c93060-cc6b-4e3f-b5a6-db6f6a9859da None admin] RBAC: Adding query filter params (name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:47.162844 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c93060-cc6b-4e3f-b5a6-db6f6a9859da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:47.164197 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c93060-cc6b-4e3f-b5a6-db6f6a9859da None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:47.172558 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 290/570] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:37:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:47.173335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c93060-cc6b-4e3f-b5a6-db6f6a9859da None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:47.219288 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:47.240358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c2ffa-894d-4685-828b-e901d5dafb97 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:47.278565 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2341ef6-20c0-4571-b816-0aa765b52f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:47.279175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2341ef6-20c0-4571-b816-0aa765b52f63 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:47.279345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c93060-cc6b-4e3f-b5a6-db6f6a9859da None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:47.279445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2341ef6-20c0-4571-b816-0aa765b52f63 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:47.279708 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2341ef6-20c0-4571-b816-0aa765b52f63 None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:47.280601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2341ef6-20c0-4571-b816-0aa765b52f63 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:47.280907 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2341ef6-20c0-4571-b816-0aa765b52f63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:47.282331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2341ef6-20c0-4571-b816-0aa765b52f63 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:47.300310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 281/571] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Aug 3 18:37:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:47.301116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2341ef6-20c0-4571-b816-0aa765b52f63 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:47.320456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138915c-842c-4a1c-9d91-04cfc12493f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:47.321643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138915c-842c-4a1c-9d91-04cfc12493f4 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:47.321919 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138915c-842c-4a1c-9d91-04cfc12493f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:47.322188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138915c-842c-4a1c-9d91-04cfc12493f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:47.333608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138915c-842c-4a1c-9d91-04cfc12493f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:47.335070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138915c-842c-4a1c-9d91-04cfc12493f4 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:47.350609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-f138915c-842c-4a1c-9d91-04cfc12493f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:47.350609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:47.350609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:47.350609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:47.350609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:47.350609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:47.350609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:47.350609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:47.350609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:47.350609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:47.350609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:47.350609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:47.350609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:47.350609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:47.350609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:47.351679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:47.353500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:47.353500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:47.353500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:47.353500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:47.353500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 291/572] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:47.353500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a8171-67b3-4860-8572-497671fb4565 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:47.353500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a8171-67b3-4860-8572-497671fb4565 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:47.353500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a8171-67b3-4860-8572-497671fb4565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:47.371758 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-9f1a8171-67b3-4860-8572-497671fb4565 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:47.372979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-9f1a8171-67b3-4860-8572-497671fb4565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:47.410603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:47.413229 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2341ef6-20c0-4571-b816-0aa765b52f63 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:47.443551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 292/573] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:47.444420 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a8171-67b3-4860-8572-497671fb4565 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:47.455761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f196d-3c04-429a-aaf3-be48497f0d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:47.456720 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f196d-3c04-429a-aaf3-be48497f0d72 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:47.457010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f196d-3c04-429a-aaf3-be48497f0d72 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:47.457446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f196d-3c04-429a-aaf3-be48497f0d72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:47.458238 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f196d-3c04-429a-aaf3-be48497f0d72 None admin] RBAC: Adding query filter params (role.id=d67cdb8bae3648fd8f3589ab33176d14, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=96ed059287274ca392c6abb893551094) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:47.458552 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f196d-3c04-429a-aaf3-be48497f0d72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:47.459985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f196d-3c04-429a-aaf3-be48497f0d72 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:47.476406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 282/574] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:47 2022] GET /identity/v3/role_assignments?user.id=96ed059287274ca392c6abb893551094&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=d67cdb8bae3648fd8f3589ab33176d14 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:47.476406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f196d-3c04-429a-aaf3-be48497f0d72 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:47.476406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:47.565031 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a8171-67b3-4860-8572-497671fb4565 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:47.587878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f196d-3c04-429a-aaf3-be48497f0d72 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:47.606487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11534ba1-ec46-4ce3-94d8-48ce69e7062b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:47.607169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11534ba1-ec46-4ce3-94d8-48ce69e7062b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:47.607455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11534ba1-ec46-4ce3-94d8-48ce69e7062b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:47.607737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11534ba1-ec46-4ce3-94d8-48ce69e7062b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:47.619661 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11534ba1-ec46-4ce3-94d8-48ce69e7062b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:47.621190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11534ba1-ec46-4ce3-94d8-48ce69e7062b None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:47.629924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f260221-bb18-4124-8af7-ad63f4b80db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:47.630820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f260221-bb18-4124-8af7-ad63f4b80db6 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:47.631106 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f260221-bb18-4124-8af7-ad63f4b80db6 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:47.631385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f260221-bb18-4124-8af7-ad63f4b80db6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:47.639623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-11534ba1-ec46-4ce3-94d8-48ce69e7062b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:47.639623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:47.639623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:47.639623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:47.639623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:47.639623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:47.639623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:47.639623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:47.639623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:47.639623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:47.639623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:47.639623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:47.639623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:47.639623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:47.639623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:47.645560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:47.646463 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:47.646463 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:47.646463 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:47.646463 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:47.646463 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 293/575] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:47.646463 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11534ba1-ec46-4ce3-94d8-48ce69e7062b None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:47.651760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f260221-bb18-4124-8af7-ad63f4b80db6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:47.653355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f260221-bb18-4124-8af7-ad63f4b80db6 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:47.668952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-1f260221-bb18-4124-8af7-ad63f4b80db6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:47.668952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:47.668952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:47.668952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:47.668952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:47.668952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:47.668952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:47.668952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:47.668952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:47.668952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:47.668952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:47.668952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:47.668952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:47.668952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:47.668952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:47.669868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:47.670792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:47.670792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:47.670792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:47.670792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:47.670792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 283/576] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:47.671075 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f260221-bb18-4124-8af7-ad63f4b80db6 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:47.677627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:47.731478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:47.762493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11534ba1-ec46-4ce3-94d8-48ce69e7062b None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:47.781805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f260221-bb18-4124-8af7-ad63f4b80db6 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:47.800996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409fbf9-70e0-4298-9148-8748356b1997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:47.801612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409fbf9-70e0-4298-9148-8748356b1997 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:47.801892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409fbf9-70e0-4298-9148-8748356b1997 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:47.802170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409fbf9-70e0-4298-9148-8748356b1997 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:47.802878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409fbf9-70e0-4298-9148-8748356b1997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:47.803180 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409fbf9-70e0-4298-9148-8748356b1997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:47.804598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409fbf9-70e0-4298-9148-8748356b1997 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:47.817152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 294/577] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:47.818683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5affaa0-9f2a-435f-8e29-869b921d94fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:47.819336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5affaa0-9f2a-435f-8e29-869b921d94fb None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:47.819622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5affaa0-9f2a-435f-8e29-869b921d94fb None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:47.819898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5affaa0-9f2a-435f-8e29-869b921d94fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:47.822438 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409fbf9-70e0-4298-9148-8748356b1997 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:47.831235 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5affaa0-9f2a-435f-8e29-869b921d94fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-d5affaa0-9f2a-435f-8e29-869b921d94fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:47.836917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:47.838073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:47.839044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:47.839044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:47.839044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:47.839044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:47.839044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:47.839044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:47.839044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:47.839044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:47.839044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:47.839044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:47.839044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:47.839044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:47.839044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 284/578] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:47.839044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5affaa0-9f2a-435f-8e29-869b921d94fb None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:47.926895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:47.947869 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409fbf9-70e0-4298-9148-8748356b1997 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:47.958475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5affaa0-9f2a-435f-8e29-869b921d94fb None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:47.973858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:47.988461 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fc91b-3f79-4841-8e9b-1c2da3ffa70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:47.989020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fc91b-3f79-4841-8e9b-1c2da3ffa70f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:47.989406 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fc91b-3f79-4841-8e9b-1c2da3ffa70f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:47.989690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fc91b-3f79-4841-8e9b-1c2da3ffa70f None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:47.990425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7fc91b-3f79-4841-8e9b-1c2da3ffa70f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:47.990740 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7fc91b-3f79-4841-8e9b-1c2da3ffa70f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:47.992124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7fc91b-3f79-4841-8e9b-1c2da3ffa70f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:47.995966 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941232a5-0555-4a07-9285-2b1d18052c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:47.996624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941232a5-0555-4a07-9285-2b1d18052c8c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:47.996906 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941232a5-0555-4a07-9285-2b1d18052c8c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:47.997179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941232a5-0555-4a07-9285-2b1d18052c8c None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:47.997880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941232a5-0555-4a07-9285-2b1d18052c8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:47.998186 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941232a5-0555-4a07-9285-2b1d18052c8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:47.999585 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941232a5-0555-4a07-9285-2b1d18052c8c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:48.010267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 295/579] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:48.010267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fc91b-3f79-4841-8e9b-1c2da3ffa70f None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:48.016668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 285/580] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:48.021627 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941232a5-0555-4a07-9285-2b1d18052c8c None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:48.117313 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fc91b-3f79-4841-8e9b-1c2da3ffa70f None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:48.132742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941232a5-0555-4a07-9285-2b1d18052c8c None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:48.157761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe1e65-23f6-4365-bf19-d74b4d929795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:48.158469 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe1e65-23f6-4365-bf19-d74b4d929795 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:48.158755 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe1e65-23f6-4365-bf19-d74b4d929795 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:48.159039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe1e65-23f6-4365-bf19-d74b4d929795 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:48.167601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe1e65-23f6-4365-bf19-d74b4d929795 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:48.169088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe1e65-23f6-4365-bf19-d74b4d929795 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:48.169720 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5f0cc-7584-468a-8bef-cd6d7f484ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:48.170354 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5f0cc-7584-468a-8bef-cd6d7f484ec8 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:48.170642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5f0cc-7584-468a-8bef-cd6d7f484ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:48.170922 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5f0cc-7584-468a-8bef-cd6d7f484ec8 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:48.171646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5f0cc-7584-468a-8bef-cd6d7f484ec8 None admin] RBAC: Adding query filter params (name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:48.172281 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5f0cc-7584-468a-8bef-cd6d7f484ec8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:48.173346 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5f0cc-7584-468a-8bef-cd6d7f484ec8 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-26fe1e65-23f6-4365-bf19-d74b4d929795 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:48.182089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:48.183101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:48.184086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:48.184086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:48.184086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:48.184086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:48.184086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:48.184086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:48.184086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:48.184086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:48.184086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 03 18:37:48.184086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:48.187518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 286/581] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:37:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:48.187627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 296/582] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:48.188368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5f0cc-7584-468a-8bef-cd6d7f484ec8 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:48.188472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe1e65-23f6-4365-bf19-d74b4d929795 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:48.193206 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:48.231289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:48.304619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe1e65-23f6-4365-bf19-d74b4d929795 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:48.311904 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5f0cc-7584-468a-8bef-cd6d7f484ec8 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:48.340779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a33f0d-4a85-42bc-9e05-9addd6c56d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:48.341419 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a33f0d-4a85-42bc-9e05-9addd6c56d6c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:48.341709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a33f0d-4a85-42bc-9e05-9addd6c56d6c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:48.341983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a33f0d-4a85-42bc-9e05-9addd6c56d6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:48.347075 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f0f7a-d92a-4779-b31d-038a6d858dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:48.347714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f0f7a-d92a-4779-b31d-038a6d858dbf None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:48.348003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f0f7a-d92a-4779-b31d-038a6d858dbf None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:48.348269 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f0f7a-d92a-4779-b31d-038a6d858dbf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:48.349165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a33f0d-4a85-42bc-9e05-9addd6c56d6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:48.350654 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a33f0d-4a85-42bc-9e05-9addd6c56d6c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:48.357302 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f0f7a-d92a-4779-b31d-038a6d858dbf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:48.358771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f0f7a-d92a-4779-b31d-038a6d858dbf None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:48.363320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-76a33f0d-4a85-42bc-9e05-9addd6c56d6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:48.363320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:48.363320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:48.363320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:48.363320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:48.363320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:48.363320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:48.363320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:48.363320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:48.363320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:48.363320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:48.363320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:48.363320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:37:48.363320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:37:48.363320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:48.364373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:48.370984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:37:48.370984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:37:48.370984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:37:48.370984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:37:48.370984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:37:48.370984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:37:48.370984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:48.370984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:48.370984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 297/583] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:48.370984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a33f0d-4a85-42bc-9e05-9addd6c56d6c None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-654f0f7a-d92a-4779-b31d-038a6d858dbf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:48.377202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:48.378144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:48.379361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:48.379361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:48.379361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:48.379361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:48.379361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:48.379361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:48.379361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:48.379361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:48.379361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 18:37:48.379361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:48.379361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 287/584] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:48.379965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f0f7a-d92a-4779-b31d-038a6d858dbf None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:48.441211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:48.483634 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:48.486572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a33f0d-4a85-42bc-9e05-9addd6c56d6c None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:48.490984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f0f7a-d92a-4779-b31d-038a6d858dbf None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:48.524703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e584447-ea91-4b73-a3a9-b6270b16f469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:48.525400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e584447-ea91-4b73-a3a9-b6270b16f469 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:48.525933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e584447-ea91-4b73-a3a9-b6270b16f469 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:48.526165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e584447-ea91-4b73-a3a9-b6270b16f469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:48.527704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb88a9d-ae9d-4955-b5eb-1176bef9d817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:48.528431 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb88a9d-ae9d-4955-b5eb-1176bef9d817 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:48.528719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb88a9d-ae9d-4955-b5eb-1176bef9d817 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:48.528998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb88a9d-ae9d-4955-b5eb-1176bef9d817 None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:48.529807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb88a9d-ae9d-4955-b5eb-1176bef9d817 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:48.530116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb88a9d-ae9d-4955-b5eb-1176bef9d817 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:48.531581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb88a9d-ae9d-4955-b5eb-1176bef9d817 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:48.537992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e584447-ea91-4b73-a3a9-b6270b16f469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:48.540603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e584447-ea91-4b73-a3a9-b6270b16f469 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:48.551073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 288/585] 10.222.0.69 () {56 vars in 1137 bytes} [Wed Aug 3 18:37:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:48.551996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb88a9d-ae9d-4955-b5eb-1176bef9d817 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:48.557318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-6e584447-ea91-4b73-a3a9-b6270b16f469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:48.557318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:48.557318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:48.557318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:48.557318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:48.557318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:48.557318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:48.557318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:48.557318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:48.557318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:48.557318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:48.557318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:48.557318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:48.557318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:48.557318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:48.558417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:48.559340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:48.559340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:48.559340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:48.559340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:48.561289 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 298/586] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:48.562275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e584447-ea91-4b73-a3a9-b6270b16f469 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:48.666279 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb88a9d-ae9d-4955-b5eb-1176bef9d817 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:48.680212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e584447-ea91-4b73-a3a9-b6270b16f469 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:48.698263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:48.702531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0b079-b9ca-46ae-8c26-91cfc4b465fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:48.703162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0b079-b9ca-46ae-8c26-91cfc4b465fd None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:48.703448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0b079-b9ca-46ae-8c26-91cfc4b465fd None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:48.703725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0b079-b9ca-46ae-8c26-91cfc4b465fd None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:48.704524 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0b079-b9ca-46ae-8c26-91cfc4b465fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:48.704835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0b079-b9ca-46ae-8c26-91cfc4b465fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:48.706290 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0b079-b9ca-46ae-8c26-91cfc4b465fd None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:48.726571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 289/587] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:37:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:48.726806 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0257a-37a7-4127-937c-6f4c2466da5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:48.727400 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0b079-b9ca-46ae-8c26-91cfc4b465fd None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:48.727479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0257a-37a7-4127-937c-6f4c2466da5a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:48.727479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0257a-37a7-4127-937c-6f4c2466da5a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:48.727479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0257a-37a7-4127-937c-6f4c2466da5a None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:48.728360 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0257a-37a7-4127-937c-6f4c2466da5a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:48.728742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0257a-37a7-4127-937c-6f4c2466da5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:48.730206 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0257a-37a7-4127-937c-6f4c2466da5a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:48.742804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:48.749156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 299/588] 10.222.0.69 () {56 vars in 1137 bytes} [Wed Aug 3 18:37:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:48.749861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0257a-37a7-4127-937c-6f4c2466da5a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:48.835088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0b079-b9ca-46ae-8c26-91cfc4b465fd None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:48.860178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0257a-37a7-4127-937c-6f4c2466da5a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:48.870691 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb47088-8618-44e4-bd7c-9374ea42ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:48.871339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb47088-8618-44e4-bd7c-9374ea42ba19 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:48.871621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb47088-8618-44e4-bd7c-9374ea42ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:48.871898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb47088-8618-44e4-bd7c-9374ea42ba19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:48.883292 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb47088-8618-44e4-bd7c-9374ea42ba19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:48.885022 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb47088-8618-44e4-bd7c-9374ea42ba19 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:48.898508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68263ad5-6b53-4954-ba7a-c7929e28ae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:48.899770 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68263ad5-6b53-4954-ba7a-c7929e28ae6a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:48.900121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68263ad5-6b53-4954-ba7a-c7929e28ae6a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:48.900427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68263ad5-6b53-4954-ba7a-c7929e28ae6a None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:48.900732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-afb47088-8618-44e4-bd7c-9374ea42ba19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:48.900732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:48.900732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:48.900732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:48.900732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:48.900732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:48.900732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:48.900732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:48.900732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:48.900732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:48.900732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:48.900732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:48.900732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:48.900732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:48.900732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:48.901668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:48.902584 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:48.902584 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:48.902584 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:48.902584 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:48.902804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68263ad5-6b53-4954-ba7a-c7929e28ae6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:48.902804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68263ad5-6b53-4954-ba7a-c7929e28ae6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:48.902804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68263ad5-6b53-4954-ba7a-c7929e28ae6a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:48.906658 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 290/589] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:48.907480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb47088-8618-44e4-bd7c-9374ea42ba19 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:48.919821 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 300/590] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:48.920604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68263ad5-6b53-4954-ba7a-c7929e28ae6a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:48.959738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:48.987626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:49.034102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb47088-8618-44e4-bd7c-9374ea42ba19 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:49.034843 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68263ad5-6b53-4954-ba7a-c7929e28ae6a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:49.075330 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31161e86-b429-4040-a80e-d547fe1131ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:49.076168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31161e86-b429-4040-a80e-d547fe1131ae None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:49.076580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31161e86-b429-4040-a80e-d547fe1131ae None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:49.076865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31161e86-b429-4040-a80e-d547fe1131ae None admin] PATH_INFO: `/v3/projects/59f00389bb404b4392d98e22e045c84f/users/850a50cfe4394febbafb0908b08888b2/roles/d67cdb8bae3648fd8f3589ab33176d14` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:49.077359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d4133-c62a-4093-abdf-db9f796f5e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:49.078337 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d4133-c62a-4093-abdf-db9f796f5e6d None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:49.078782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d4133-c62a-4093-abdf-db9f796f5e6d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:49.079172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d4133-c62a-4093-abdf-db9f796f5e6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:49.093144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31161e86-b429-4040-a80e-d547fe1131ae None admin] RBAC: Authorizing `identity:create_grant(project_id=59f00389bb404b4392d98e22e045c84f, user_id=850a50cfe4394febbafb0908b08888b2, role_id=d67cdb8bae3648fd8f3589ab33176d14)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:49.093841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d4133-c62a-4093-abdf-db9f796f5e6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:49.094507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31161e86-b429-4040-a80e-d547fe1131ae None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:49.095289 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d4133-c62a-4093-abdf-db9f796f5e6d None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:49.109714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 291/591] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:37:48 2022] PUT /identity/v3/projects/59f00389bb404b4392d98e22e045c84f/users/850a50cfe4394febbafb0908b08888b2/roles/d67cdb8bae3648fd8f3589ab33176d14 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:37:49.110851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31161e86-b429-4040-a80e-d547fe1131ae None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:49.111388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-c36d4133-c62a-4093-abdf-db9f796f5e6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:49.111388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:49.111388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:49.111388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:49.111388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:49.111388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:49.111388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:49.111388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:49.111388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:49.111388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:49.111388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:49.111388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:49.111388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:49.111388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:49.111388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:49.112270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:49.113189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:49.113189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:49.113189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:49.113189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:49.117821 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 301/592] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:49.118822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d4133-c62a-4093-abdf-db9f796f5e6d None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:49.245189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:49.255454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31161e86-b429-4040-a80e-d547fe1131ae None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:49.269229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d4133-c62a-4093-abdf-db9f796f5e6d None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:49.279516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:49.308192 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434784e-ffa7-4a38-8625-3004d4e7a910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:49.308888 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434784e-ffa7-4a38-8625-3004d4e7a910 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:49.309185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434784e-ffa7-4a38-8625-3004d4e7a910 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:49.309455 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434784e-ffa7-4a38-8625-3004d4e7a910 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:49.310160 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434784e-ffa7-4a38-8625-3004d4e7a910 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:49.310581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434784e-ffa7-4a38-8625-3004d4e7a910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:49.312076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434784e-ffa7-4a38-8625-3004d4e7a910 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:49.320274 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd982411-fa19-4d72-9d2f-2565bb884d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:49.321174 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd982411-fa19-4d72-9d2f-2565bb884d3f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:49.321911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd982411-fa19-4d72-9d2f-2565bb884d3f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:49.321911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd982411-fa19-4d72-9d2f-2565bb884d3f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:49.325278 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 292/593] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:49.325963 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434784e-ffa7-4a38-8625-3004d4e7a910 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:49.334114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd982411-fa19-4d72-9d2f-2565bb884d3f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:49.335581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd982411-fa19-4d72-9d2f-2565bb884d3f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-fd982411-fa19-4d72-9d2f-2565bb884d3f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:49.347910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:49.348908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:49.349882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:49.349882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:49.349882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:49.349882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:49.349882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:49.349882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:49.349882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:49.349882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:49.349882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 18:37:49.349882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:49.349882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 302/594] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:49.349882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd982411-fa19-4d72-9d2f-2565bb884d3f None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:49.464150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434784e-ffa7-4a38-8625-3004d4e7a910 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:49.487343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd982411-fa19-4d72-9d2f-2565bb884d3f None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:49.508521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1e0b2-b431-423a-801a-e916d492ca8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:49.509323 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1e0b2-b431-423a-801a-e916d492ca8c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:49.509696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1e0b2-b431-423a-801a-e916d492ca8c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:49.510064 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1e0b2-b431-423a-801a-e916d492ca8c None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:49.510859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1e0b2-b431-423a-801a-e916d492ca8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:49.511708 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1e0b2-b431-423a-801a-e916d492ca8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:49.513495 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1e0b2-b431-423a-801a-e916d492ca8c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:49.527009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b62e55-1e32-45df-bdd6-1c2f5d8352f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:49.527947 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b62e55-1e32-45df-bdd6-1c2f5d8352f8 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:49.528467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b62e55-1e32-45df-bdd6-1c2f5d8352f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:49.528924 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b62e55-1e32-45df-bdd6-1c2f5d8352f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:49.530102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 293/595] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:49.531113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1e0b2-b431-423a-801a-e916d492ca8c None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:49.537972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b62e55-1e32-45df-bdd6-1c2f5d8352f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:49.540276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b62e55-1e32-45df-bdd6-1c2f5d8352f8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:49.553595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-b8b62e55-1e32-45df-bdd6-1c2f5d8352f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:49.553595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:49.553595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:49.553595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:49.553595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:49.553595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:49.553595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:49.553595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:49.553595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:49.553595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:49.553595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:49.553595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:49.553595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:37:49.553595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:37:49.553595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:49.554552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:49.555486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:37:49.555486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:37:49.555486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:37:49.555486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:37:49.555486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:37:49.555486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:37:49.555486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:49.555486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:49.557820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 303/596] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:49.558827 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b62e55-1e32-45df-bdd6-1c2f5d8352f8 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:49.575612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:49.578010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:49.647500 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1e0b2-b431-423a-801a-e916d492ca8c None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:49.684575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d16ee-afc8-4da4-ba2b-c5d2f9acb53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:49.685364 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d16ee-afc8-4da4-ba2b-c5d2f9acb53d None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:49.685948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d16ee-afc8-4da4-ba2b-c5d2f9acb53d None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:49.686237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d16ee-afc8-4da4-ba2b-c5d2f9acb53d None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:49.687203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d16ee-afc8-4da4-ba2b-c5d2f9acb53d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:49.687514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d16ee-afc8-4da4-ba2b-c5d2f9acb53d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:49.688977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d16ee-afc8-4da4-ba2b-c5d2f9acb53d None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:49.689087 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b62e55-1e32-45df-bdd6-1c2f5d8352f8 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:49.709061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 294/597] 10.222.0.69 () {56 vars in 1139 bytes} [Wed Aug 3 18:37:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:49.709982 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d16ee-afc8-4da4-ba2b-c5d2f9acb53d None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:49.732860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143b160-1ea9-44dd-bbd6-fb7af88ae432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:49.733814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143b160-1ea9-44dd-bbd6-fb7af88ae432 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:49.734275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143b160-1ea9-44dd-bbd6-fb7af88ae432 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:49.734727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143b160-1ea9-44dd-bbd6-fb7af88ae432 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:49.742539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143b160-1ea9-44dd-bbd6-fb7af88ae432 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:49.744055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143b160-1ea9-44dd-bbd6-fb7af88ae432 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:49.755788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-4143b160-1ea9-44dd-bbd6-fb7af88ae432 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:49.755788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:49.755788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:49.755788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:49.755788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:49.755788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:49.755788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:49.755788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:49.755788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:49.755788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:49.755788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:49.755788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:49.755788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:37:49.755788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:37:49.755788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:49.756836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:49.757747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:37:49.757747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:37:49.757747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:37:49.757747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:37:49.757747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:37:49.757747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:37:49.757747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:49.757747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:49.757747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 304/598] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:49.759335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579e223-ad8f-42f9-afcb-1e864270a024 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:49.759440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579e223-ad8f-42f9-afcb-1e864270a024 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:49.759720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579e223-ad8f-42f9-afcb-1e864270a024 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:49.761156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 305/599] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:49.761728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579e223-ad8f-42f9-afcb-1e864270a024 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:49.826487 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:49.834902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d16ee-afc8-4da4-ba2b-c5d2f9acb53d None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:49.842717 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:49.875614 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d419896-3e7e-4675-84ad-5259da235c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:49.876393 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d419896-3e7e-4675-84ad-5259da235c8a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:49.876679 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d419896-3e7e-4675-84ad-5259da235c8a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:49.876955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d419896-3e7e-4675-84ad-5259da235c8a None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:49.877712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d419896-3e7e-4675-84ad-5259da235c8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:49.878017 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d419896-3e7e-4675-84ad-5259da235c8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:49.879705 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d419896-3e7e-4675-84ad-5259da235c8a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:49.883659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579e223-ad8f-42f9-afcb-1e864270a024 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:49.897307 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 295/600] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:37:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:49.897307 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d419896-3e7e-4675-84ad-5259da235c8a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:49.927950 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e820d3-fd54-4439-a89a-51de35f627e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:49.928982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e820d3-fd54-4439-a89a-51de35f627e1 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:49.929253 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e820d3-fd54-4439-a89a-51de35f627e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:49.929520 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e820d3-fd54-4439-a89a-51de35f627e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:49.940511 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e820d3-fd54-4439-a89a-51de35f627e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:49.941987 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e820d3-fd54-4439-a89a-51de35f627e1 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:49.958339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-20e820d3-fd54-4439-a89a-51de35f627e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:49.958339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:49.958339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:49.958339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:49.958339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:49.958339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:49.958339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:49.958339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:49.958339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:49.958339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:49.958339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:49.958339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:49.958339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:49.958339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:49.958339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:49.959445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:49.960567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:49.960567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:49.960567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:49.960567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:49.964813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 306/601] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:49.966126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3034f-3382-4300-9cf6-ff65008121f1 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:49.966422 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3034f-3382-4300-9cf6-ff65008121f1 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:49.966701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3034f-3382-4300-9cf6-ff65008121f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:49.985180 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-51e3034f-3382-4300-9cf6-ff65008121f1 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:49.986477 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-51e3034f-3382-4300-9cf6-ff65008121f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:50.014018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d419896-3e7e-4675-84ad-5259da235c8a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:50.051437 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb12362-53a1-4f65-9c88-9cb48c625e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:50.052056 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb12362-53a1-4f65-9c88-9cb48c625e10 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:50.052365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb12362-53a1-4f65-9c88-9cb48c625e10 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:50.052646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb12362-53a1-4f65-9c88-9cb48c625e10 None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:50.053406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb12362-53a1-4f65-9c88-9cb48c625e10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:50.053712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb12362-53a1-4f65-9c88-9cb48c625e10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:50.055197 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb12362-53a1-4f65-9c88-9cb48c625e10 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:50.064836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 307/602] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:50.064836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3034f-3382-4300-9cf6-ff65008121f1 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:50.072784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 296/603] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:37:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:50.077912 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb12362-53a1-4f65-9c88-9cb48c625e10 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:50.094777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:50.124454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:50.177561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3034f-3382-4300-9cf6-ff65008121f1 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:50.192462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb12362-53a1-4f65-9c88-9cb48c625e10 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:50.216216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad3853-eeb7-4d77-adf0-df94820401ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:50.216928 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad3853-eeb7-4d77-adf0-df94820401ab None admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:50.217213 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad3853-eeb7-4d77-adf0-df94820401ab None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:50.217568 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad3853-eeb7-4d77-adf0-df94820401ab None admin] PATH_INFO: `/v3/projects/59f00389bb404b4392d98e22e045c84f/users/4799b0c9e5aa47d389121d2183666d3d/roles/d67cdb8bae3648fd8f3589ab33176d14` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:50.230149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689619b9-1372-4758-a274-77b5f27ecb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:50.230334 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689619b9-1372-4758-a274-77b5f27ecb80 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:50.230586 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689619b9-1372-4758-a274-77b5f27ecb80 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:50.230859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689619b9-1372-4758-a274-77b5f27ecb80 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:50.231558 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689619b9-1372-4758-a274-77b5f27ecb80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:50.231865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689619b9-1372-4758-a274-77b5f27ecb80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:50.233261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689619b9-1372-4758-a274-77b5f27ecb80 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:50.235334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad3853-eeb7-4d77-adf0-df94820401ab None admin] RBAC: Authorizing `identity:create_grant(project_id=59f00389bb404b4392d98e22e045c84f, user_id=4799b0c9e5aa47d389121d2183666d3d, role_id=d67cdb8bae3648fd8f3589ab33176d14)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:50.236829 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad3853-eeb7-4d77-adf0-df94820401ab None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:50.251299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 297/604] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:50.251299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689619b9-1372-4758-a274-77b5f27ecb80 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:50.259655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 308/605] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:37:50 2022] PUT /identity/v3/projects/59f00389bb404b4392d98e22e045c84f/users/4799b0c9e5aa47d389121d2183666d3d/roles/d67cdb8bae3648fd8f3589ab33176d14 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:37:50.259655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad3853-eeb7-4d77-adf0-df94820401ab None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:50.334954 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:50.376333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689619b9-1372-4758-a274-77b5f27ecb80 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:50.400872 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:50.406311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad3853-eeb7-4d77-adf0-df94820401ab None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:50.432839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03f1ee-e4ed-4a26-9eac-537e4369e2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:50.433771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03f1ee-e4ed-4a26-9eac-537e4369e2d6 None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:50.434198 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03f1ee-e4ed-4a26-9eac-537e4369e2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:50.435758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03f1ee-e4ed-4a26-9eac-537e4369e2d6 None admin] PATH_INFO: `/v3/projects/59f00389bb404b4392d98e22e045c84f/users/c7d0fde945184c93b8fd91d15b66f534/roles/d67cdb8bae3648fd8f3589ab33176d14` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:50.452594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ddbaa-8ee9-4ed2-905a-d2da3ff9cbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:50.453394 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ddbaa-8ee9-4ed2-905a-d2da3ff9cbb8 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:50.453802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ddbaa-8ee9-4ed2-905a-d2da3ff9cbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:50.454177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ddbaa-8ee9-4ed2-905a-d2da3ff9cbb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:50.456790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03f1ee-e4ed-4a26-9eac-537e4369e2d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=59f00389bb404b4392d98e22e045c84f, user_id=c7d0fde945184c93b8fd91d15b66f534, role_id=d67cdb8bae3648fd8f3589ab33176d14)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:50.458631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03f1ee-e4ed-4a26-9eac-537e4369e2d6 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:50.467691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ddbaa-8ee9-4ed2-905a-d2da3ff9cbb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-0d5ddbaa-8ee9-4ed2-905a-d2da3ff9cbb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:50.473540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:50.474619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:50.475639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:50.475639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:50.475639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:50.475639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:50.475639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:50.475639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:50.475639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:50.475639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:50.475639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:50.475639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:50.475639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:50.475639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:50.478401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 309/606] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:50.479464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ddbaa-8ee9-4ed2-905a-d2da3ff9cbb8 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:50.483361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 298/607] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:37:50 2022] PUT /identity/v3/projects/59f00389bb404b4392d98e22e045c84f/users/c7d0fde945184c93b8fd91d15b66f534/roles/d67cdb8bae3648fd8f3589ab33176d14 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:37:50.484357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03f1ee-e4ed-4a26-9eac-537e4369e2d6 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:50.651718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ddbaa-8ee9-4ed2-905a-d2da3ff9cbb8 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:50.665594 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03f1ee-e4ed-4a26-9eac-537e4369e2d6 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:50.715542 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:50.729279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace48e0d-0acd-4669-b2a5-8b45a470b098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:50.729798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace48e0d-0acd-4669-b2a5-8b45a470b098 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:50.729798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace48e0d-0acd-4669-b2a5-8b45a470b098 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:50.729963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace48e0d-0acd-4669-b2a5-8b45a470b098 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:50.736682 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1a899-fb47-4a04-bd05-fa5ba4a59c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:50.737982 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1a899-fb47-4a04-bd05-fa5ba4a59c94 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:50.738517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1a899-fb47-4a04-bd05-fa5ba4a59c94 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:50.738983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1a899-fb47-4a04-bd05-fa5ba4a59c94 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:50.739106 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace48e0d-0acd-4669-b2a5-8b45a470b098 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:50.740136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1a899-fb47-4a04-bd05-fa5ba4a59c94 None admin] RBAC: Adding query filter params (name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:50.740616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace48e0d-0acd-4669-b2a5-8b45a470b098 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:50.740910 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1a899-fb47-4a04-bd05-fa5ba4a59c94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:50.744494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1a899-fb47-4a04-bd05-fa5ba4a59c94 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:50.755354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-ace48e0d-0acd-4669-b2a5-8b45a470b098 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:50.755354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:50.755354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:50.755354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:50.755354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:50.755354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:50.755354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:50.755354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:50.755354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:50.755354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:50.755354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:50.755354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:50.755354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:37:50.755354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:37:50.755354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:50.756370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:50.757280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:37:50.757280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:37:50.757280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:37:50.757280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:37:50.757280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:37:50.757280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:37:50.757280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:50.757280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:50.757280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 310/609] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:50.758728 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 299/609] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:37:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:50.762107 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace48e0d-0acd-4669-b2a5-8b45a470b098 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:50.764953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1a899-fb47-4a04-bd05-fa5ba4a59c94 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:50.790891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:50.889046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace48e0d-0acd-4669-b2a5-8b45a470b098 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:50.917177 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1a899-fb47-4a04-bd05-fa5ba4a59c94 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:50.939182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175c738-5bdc-45d6-9a99-a6094deaf700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:50.939919 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175c738-5bdc-45d6-9a99-a6094deaf700 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:50.940275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175c738-5bdc-45d6-9a99-a6094deaf700 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:50.940670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175c738-5bdc-45d6-9a99-a6094deaf700 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:50.953337 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175c738-5bdc-45d6-9a99-a6094deaf700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:50.955233 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175c738-5bdc-45d6-9a99-a6094deaf700 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:50.965382 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4fa2c-5b49-4673-924e-8350dd8885db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:50.966436 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4fa2c-5b49-4673-924e-8350dd8885db None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:50.967009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4fa2c-5b49-4673-924e-8350dd8885db None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:50.967433 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4fa2c-5b49-4673-924e-8350dd8885db None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:50.968368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4fa2c-5b49-4673-924e-8350dd8885db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:50.968832 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4fa2c-5b49-4673-924e-8350dd8885db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:50.970407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4fa2c-5b49-4673-924e-8350dd8885db None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:50.971505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-9175c738-5bdc-45d6-9a99-a6094deaf700 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:50.971505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:50.971505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:50.971505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:50.971505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:50.971505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:50.971505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:50.971505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:50.971505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:50.971505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:50.971505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:50.971505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:50.971505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:50.971505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:50.971505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:50.972608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:50.973714 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:50.973714 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:50.973714 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:50.973714 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:50.975296 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 311/610] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:50.980643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175c738-5bdc-45d6-9a99-a6094deaf700 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:50.987672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 300/611] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:37:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:50.994112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4fa2c-5b49-4673-924e-8350dd8885db None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:51.092644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:51.132426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:51.165220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4fa2c-5b49-4673-924e-8350dd8885db None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:51.187025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175c738-5bdc-45d6-9a99-a6094deaf700 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:51.233405 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74e5de-8b17-4640-8a74-c4da91f43c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:51.234470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74e5de-8b17-4640-8a74-c4da91f43c49 None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:51.235056 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74e5de-8b17-4640-8a74-c4da91f43c49 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:51.235537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74e5de-8b17-4640-8a74-c4da91f43c49 None admin] PATH_INFO: `/v3/projects/59f00389bb404b4392d98e22e045c84f/users/0eb9ad672e5444098d71b7c9a27b72e8/roles/d67cdb8bae3648fd8f3589ab33176d14` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:51.249193 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cff5dc-5192-4b02-89c5-9a86f21aaf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:51.249996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cff5dc-5192-4b02-89c5-9a86f21aaf09 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:51.250299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cff5dc-5192-4b02-89c5-9a86f21aaf09 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:51.250583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cff5dc-5192-4b02-89c5-9a86f21aaf09 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:51.251303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cff5dc-5192-4b02-89c5-9a86f21aaf09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:51.251616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cff5dc-5192-4b02-89c5-9a86f21aaf09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:51.254974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cff5dc-5192-4b02-89c5-9a86f21aaf09 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:51.275122 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 312/612] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:51.281029 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cff5dc-5192-4b02-89c5-9a86f21aaf09 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:51.287111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74e5de-8b17-4640-8a74-c4da91f43c49 None admin] RBAC: Authorizing `identity:create_grant(project_id=59f00389bb404b4392d98e22e045c84f, user_id=0eb9ad672e5444098d71b7c9a27b72e8, role_id=d67cdb8bae3648fd8f3589ab33176d14)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:51.288574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74e5de-8b17-4640-8a74-c4da91f43c49 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:51.307107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 301/613] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:37:50 2022] PUT /identity/v3/projects/59f00389bb404b4392d98e22e045c84f/users/0eb9ad672e5444098d71b7c9a27b72e8/roles/d67cdb8bae3648fd8f3589ab33176d14 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:37:51.309349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e8a13-7d27-44fd-b60a-d0fdc163a187 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:51.309771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e8a13-7d27-44fd-b60a-d0fdc163a187 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:51.310049 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e8a13-7d27-44fd-b60a-d0fdc163a187 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:51.316468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 302/614] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:51.318568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1c149-c0ab-4d4e-952b-da9c59959b38 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:51.318837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1c149-c0ab-4d4e-952b-da9c59959b38 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:51.319115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1c149-c0ab-4d4e-952b-da9c59959b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:51.343459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-92d1c149-c0ab-4d4e-952b-da9c59959b38 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:51.344725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-92d1c149-c0ab-4d4e-952b-da9c59959b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:51.468414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 303/615] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:51.487007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cff5dc-5192-4b02-89c5-9a86f21aaf09 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:51.501297 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1c149-c0ab-4d4e-952b-da9c59959b38 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:51.526329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:51.534011 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:51.535640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14846404-8359-4daa-9674-63658946f851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:51.536617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14846404-8359-4daa-9674-63658946f851 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:51.537067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14846404-8359-4daa-9674-63658946f851 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:51.537508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14846404-8359-4daa-9674-63658946f851 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:51.550110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14846404-8359-4daa-9674-63658946f851 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:51.552061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14846404-8359-4daa-9674-63658946f851 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-14846404-8359-4daa-9674-63658946f851 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:51.566832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:51.567852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:51.568865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:51.568865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:51.568865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:51.568865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:51.568865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:51.568865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:51.568865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:51.568865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:51.568865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 18:37:51.568865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:51.572592 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 313/616] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:37:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:51.578112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14846404-8359-4daa-9674-63658946f851 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:51.653102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1c149-c0ab-4d4e-952b-da9c59959b38 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:51.722297 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b039c9-d84b-43f9-bbf8-d8f7fa6b3248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:51.723061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b039c9-d84b-43f9-bbf8-d8f7fa6b3248 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:51.723250 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b039c9-d84b-43f9-bbf8-d8f7fa6b3248 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:51.723500 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b039c9-d84b-43f9-bbf8-d8f7fa6b3248 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:51.745001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b039c9-d84b-43f9-bbf8-d8f7fa6b3248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-b2b039c9-d84b-43f9-bbf8-d8f7fa6b3248 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:51.758469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:51.759822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:51.760733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:51.760733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:51.760733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:51.760733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:51.760733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:51.760733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:51.760733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:51.760733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:51.760733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:51.760733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:51.760733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:51.760733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:51.760733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 304/617] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:51.768131 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14846404-8359-4daa-9674-63658946f851 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:51.778243 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b039c9-d84b-43f9-bbf8-d8f7fa6b3248 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:51.806569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7b02e-fb1a-4c76-a6a2-4493afef47b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:51.807204 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7b02e-fb1a-4c76-a6a2-4493afef47b2 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:51.807536 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7b02e-fb1a-4c76-a6a2-4493afef47b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:51.807814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7b02e-fb1a-4c76-a6a2-4493afef47b2 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:51.808714 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7b02e-fb1a-4c76-a6a2-4493afef47b2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:51.809028 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7b02e-fb1a-4c76-a6a2-4493afef47b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:51.810488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7b02e-fb1a-4c76-a6a2-4493afef47b2 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:51.830444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 314/618] 10.222.0.69 () {56 vars in 1135 bytes} [Wed Aug 3 18:37:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:51.835647 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7b02e-fb1a-4c76-a6a2-4493afef47b2 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:51.892665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:51.939690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:51.956391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7b02e-fb1a-4c76-a6a2-4493afef47b2 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:51.974295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b039c9-d84b-43f9-bbf8-d8f7fa6b3248 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:52.022338 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dd83e-46b0-4c4b-88e1-10cb6081a06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:52.023061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dd83e-46b0-4c4b-88e1-10cb6081a06e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:52.023529 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dd83e-46b0-4c4b-88e1-10cb6081a06e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:52.024037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dd83e-46b0-4c4b-88e1-10cb6081a06e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:52.042204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f406051-5634-4fa4-8aad-0e6bb1f50951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:52.042997 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f406051-5634-4fa4-8aad-0e6bb1f50951 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:52.043401 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f406051-5634-4fa4-8aad-0e6bb1f50951 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:52.043774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f406051-5634-4fa4-8aad-0e6bb1f50951 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:52.044655 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f406051-5634-4fa4-8aad-0e6bb1f50951 None admin] RBAC: Adding query filter params (name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:52.045061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f406051-5634-4fa4-8aad-0e6bb1f50951 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:52.046856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f406051-5634-4fa4-8aad-0e6bb1f50951 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:52.047852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734dd83e-46b0-4c4b-88e1-10cb6081a06e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:52.049508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734dd83e-46b0-4c4b-88e1-10cb6081a06e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:52.072312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 305/619] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:37:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:52.072312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f406051-5634-4fa4-8aad-0e6bb1f50951 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:52.076269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-734dd83e-46b0-4c4b-88e1-10cb6081a06e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:52.076269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:52.076269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:52.076269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:52.076269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:52.076269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:52.076269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:52.076269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:52.076269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:52.076269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:52.076269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:52.076269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:52.076269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:52.076269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:52.076269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:52.077516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:52.078604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:52.078604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:52.078604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:52.078604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:52.080731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 315/620] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:52.085809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dd83e-46b0-4c4b-88e1-10cb6081a06e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:52.211994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f406051-5634-4fa4-8aad-0e6bb1f50951 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:52.284970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099e8a6-b6d1-4182-8ca4-b78d8d321b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:52.287822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099e8a6-b6d1-4182-8ca4-b78d8d321b32 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:52.288114 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099e8a6-b6d1-4182-8ca4-b78d8d321b32 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:52.288415 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099e8a6-b6d1-4182-8ca4-b78d8d321b32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:52.294995 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:52.303040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dd83e-46b0-4c4b-88e1-10cb6081a06e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:52.317459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099e8a6-b6d1-4182-8ca4-b78d8d321b32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:52.318983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099e8a6-b6d1-4182-8ca4-b78d8d321b32 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:52.339463 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:52.350793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-4099e8a6-b6d1-4182-8ca4-b78d8d321b32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:52.350793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:52.350793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:52.350793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:52.350793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:52.350793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:52.350793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:52.350793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:52.350793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:52.350793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:52.350793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:52.350793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:52.350793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:52.350793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:52.350793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:52.351752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:52.352672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:52.352672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:52.352672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:52.352672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:52.361440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 306/621] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:52.364589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099e8a6-b6d1-4182-8ca4-b78d8d321b32 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:52.370640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7262d-c897-4ca9-9650-f7e92383d59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:52.371280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7262d-c897-4ca9-9650-f7e92383d59e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:52.371561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7262d-c897-4ca9-9650-f7e92383d59e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:52.371834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7262d-c897-4ca9-9650-f7e92383d59e None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:52.376819 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7262d-c897-4ca9-9650-f7e92383d59e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:52.377141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7262d-c897-4ca9-9650-f7e92383d59e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:52.378790 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7262d-c897-4ca9-9650-f7e92383d59e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:52.396995 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 316/622] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:52.403301 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7262d-c897-4ca9-9650-f7e92383d59e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:52.487232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099e8a6-b6d1-4182-8ca4-b78d8d321b32 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:52.536647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fcee9-0216-449c-9388-80ffe550f6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:52.537196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fcee9-0216-449c-9388-80ffe550f6b2 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:52.537475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fcee9-0216-449c-9388-80ffe550f6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:52.537751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fcee9-0216-449c-9388-80ffe550f6b2 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:52.539604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6fcee9-0216-449c-9388-80ffe550f6b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:52.539940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6fcee9-0216-449c-9388-80ffe550f6b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:52.541324 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6fcee9-0216-449c-9388-80ffe550f6b2 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:52.568492 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7262d-c897-4ca9-9650-f7e92383d59e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:52.571770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 307/623] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:52.589793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fcee9-0216-449c-9388-80ffe550f6b2 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:52.623572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6485eb-c7fd-4440-bb30-d7ecf0161b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:52.624230 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6485eb-c7fd-4440-bb30-d7ecf0161b66 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:52.624540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6485eb-c7fd-4440-bb30-d7ecf0161b66 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:52.624817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6485eb-c7fd-4440-bb30-d7ecf0161b66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:52.641645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6485eb-c7fd-4440-bb30-d7ecf0161b66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:52.643187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6485eb-c7fd-4440-bb30-d7ecf0161b66 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:52.664691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-1e6485eb-c7fd-4440-bb30-d7ecf0161b66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:52.664691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:52.664691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:52.664691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:52.664691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:52.664691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:52.664691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:52.664691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:52.664691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:52.664691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:52.664691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:52.664691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:52.664691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:37:52.664691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:37:52.664691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:52.665949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:52.667345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:37:52.667345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:37:52.667345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:37:52.667345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:37:52.667345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:37:52.667345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:37:52.667345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:52.667345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:52.667345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 317/624] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:52.670221 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6485eb-c7fd-4440-bb30-d7ecf0161b66 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:52.682929 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:52.724318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:52.733557 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fcee9-0216-449c-9388-80ffe550f6b2 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:52.787749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d436a8-4efd-479a-99bc-ba06cf894982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:52.788711 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d436a8-4efd-479a-99bc-ba06cf894982 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:52.789516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d436a8-4efd-479a-99bc-ba06cf894982 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:52.789828 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d436a8-4efd-479a-99bc-ba06cf894982 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:52.799488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d436a8-4efd-479a-99bc-ba06cf894982 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:52.801140 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d436a8-4efd-479a-99bc-ba06cf894982 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-c5d436a8-4efd-479a-99bc-ba06cf894982 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:52.816541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:52.817806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:52.822593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:52.822593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:52.822593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:52.822593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:52.822593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:52.822593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:52.822593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:52.822593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:52.822593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:37:52.822593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:52.822593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 308/625] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Aug 3 18:37:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:52.822593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669afcd1-090b-4793-921e-aef36836ce45 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:52.822593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669afcd1-090b-4793-921e-aef36836ce45 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:52.825780 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669afcd1-090b-4793-921e-aef36836ce45 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:52.825780 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 309/626] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:52.827579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb07469-e9d0-421b-9e31-d01f9123db94 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:52.827935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb07469-e9d0-421b-9e31-d01f9123db94 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:52.828277 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb07469-e9d0-421b-9e31-d01f9123db94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:52.851037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ecb07469-e9d0-421b-9e31-d01f9123db94 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:52.852856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ecb07469-e9d0-421b-9e31-d01f9123db94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:52.853586 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6485eb-c7fd-4440-bb30-d7ecf0161b66 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:52.912030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07402e-74ac-41e9-b38d-dfc3b35072d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:52.913015 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07402e-74ac-41e9-b38d-dfc3b35072d4 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:52.913481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07402e-74ac-41e9-b38d-dfc3b35072d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:52.914592 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07402e-74ac-41e9-b38d-dfc3b35072d4 None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:52.915547 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07402e-74ac-41e9-b38d-dfc3b35072d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:52.916055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07402e-74ac-41e9-b38d-dfc3b35072d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:52.917911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07402e-74ac-41e9-b38d-dfc3b35072d4 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:52.940128 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 318/627] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:37:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:52.941067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07402e-74ac-41e9-b38d-dfc3b35072d4 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:52.953950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 310/628] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:37:52.955227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb07469-e9d0-421b-9e31-d01f9123db94 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:53.013759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:53.052371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:53.072517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb07469-e9d0-421b-9e31-d01f9123db94 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:53.073832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07402e-74ac-41e9-b38d-dfc3b35072d4 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:53.107843 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67c3f3-1354-404e-8841-c5a9e748b590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:53.108537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67c3f3-1354-404e-8841-c5a9e748b590 None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:53.108820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67c3f3-1354-404e-8841-c5a9e748b590 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:53.109093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67c3f3-1354-404e-8841-c5a9e748b590 None admin] PATH_INFO: `/v3/projects/59f00389bb404b4392d98e22e045c84f/users/96ed059287274ca392c6abb893551094/roles/d67cdb8bae3648fd8f3589ab33176d14` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:53.110835 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed79ad1-f940-44fc-be84-7158bcddfe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:53.111569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed79ad1-f940-44fc-be84-7158bcddfe3d None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:53.111852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed79ad1-f940-44fc-be84-7158bcddfe3d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:53.112871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed79ad1-f940-44fc-be84-7158bcddfe3d None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:53.113078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed79ad1-f940-44fc-be84-7158bcddfe3d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:53.113439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed79ad1-f940-44fc-be84-7158bcddfe3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:53.114944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed79ad1-f940-44fc-be84-7158bcddfe3d None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:53.123752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67c3f3-1354-404e-8841-c5a9e748b590 None admin] RBAC: Authorizing `identity:create_grant(project_id=59f00389bb404b4392d98e22e045c84f, user_id=96ed059287274ca392c6abb893551094, role_id=d67cdb8bae3648fd8f3589ab33176d14)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:53.125217 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67c3f3-1354-404e-8841-c5a9e748b590 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:53.136128 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 319/629] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Aug 3 18:37:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:53.137254 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed79ad1-f940-44fc-be84-7158bcddfe3d None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:53.143956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 311/630] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:37:52 2022] PUT /identity/v3/projects/59f00389bb404b4392d98e22e045c84f/users/96ed059287274ca392c6abb893551094/roles/d67cdb8bae3648fd8f3589ab33176d14 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:37:53.145423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09cef3-14fd-4980-9f79-15838100af2f None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:53.145875 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09cef3-14fd-4980-9f79-15838100af2f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:53.146339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09cef3-14fd-4980-9f79-15838100af2f None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:53.147784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 312/631] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:53.149445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09cef3-14fd-4980-9f79-15838100af2f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:53.281670 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:53.290601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09cef3-14fd-4980-9f79-15838100af2f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:53.290800 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed79ad1-f940-44fc-be84-7158bcddfe3d None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:53.332731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1381841-df2c-4032-b7ca-ff0e5a99824a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:53.333307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1381841-df2c-4032-b7ca-ff0e5a99824a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:53.333440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e2143-7543-45a1-b542-b9525f864c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:53.333652 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1381841-df2c-4032-b7ca-ff0e5a99824a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:53.333860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1381841-df2c-4032-b7ca-ff0e5a99824a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:53.333959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e2143-7543-45a1-b542-b9525f864c36 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:53.334238 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e2143-7543-45a1-b542-b9525f864c36 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:53.334513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e2143-7543-45a1-b542-b9525f864c36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:53.347518 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:53.353128 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1381841-df2c-4032-b7ca-ff0e5a99824a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-a1381841-df2c-4032-b7ca-ff0e5a99824a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:53.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:53.361053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:53.362572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:53.362572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:53.362572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:53.362572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:53.362572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:53.362572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:53.362572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:53.362572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:53.362572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:53.362572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:53.362572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:53.362572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:53.362572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 320/632] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:53.362572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec602db-98a6-4444-b81a-46d770cefef9 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:53.363352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec602db-98a6-4444-b81a-46d770cefef9 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:53.363352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec602db-98a6-4444-b81a-46d770cefef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:53.387083 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-aec602db-98a6-4444-b81a-46d770cefef9 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:53.388955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-aec602db-98a6-4444-b81a-46d770cefef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:53.424144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e2143-7543-45a1-b542-b9525f864c36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:53.426058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e2143-7543-45a1-b542-b9525f864c36 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:53.463310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 321/633] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:53.464201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec602db-98a6-4444-b81a-46d770cefef9 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:53.546321 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:53.557964 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 313/634] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:53.559298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e2143-7543-45a1-b542-b9525f864c36 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:53.572484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec602db-98a6-4444-b81a-46d770cefef9 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:53.602686 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:53.612972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ce3cf-e6b0-4bbf-b9e0-fafb493dd5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:53.613592 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ce3cf-e6b0-4bbf-b9e0-fafb493dd5d7 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:53.613870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ce3cf-e6b0-4bbf-b9e0-fafb493dd5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:53.614140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ce3cf-e6b0-4bbf-b9e0-fafb493dd5d7 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:53.614861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ce3cf-e6b0-4bbf-b9e0-fafb493dd5d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:53.615161 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ce3cf-e6b0-4bbf-b9e0-fafb493dd5d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:53.616557 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ce3cf-e6b0-4bbf-b9e0-fafb493dd5d7 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:53.626166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 322/635] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:37:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:53.627062 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ce3cf-e6b0-4bbf-b9e0-fafb493dd5d7 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:53.685855 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e2143-7543-45a1-b542-b9525f864c36 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:53.722752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a067ca-2c3c-4cd8-a44f-c5dadaea5730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:53.723875 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a067ca-2c3c-4cd8-a44f-c5dadaea5730 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:53.724400 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a067ca-2c3c-4cd8-a44f-c5dadaea5730 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:53.724901 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a067ca-2c3c-4cd8-a44f-c5dadaea5730 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:53.737287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a067ca-2c3c-4cd8-a44f-c5dadaea5730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-09a067ca-2c3c-4cd8-a44f-c5dadaea5730 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:53.743054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:53.744074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:53.744991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:53.744991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:53.744991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:53.744991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:53.744991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:53.744991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:53.744991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:53.744991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:53.744991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:53.744991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:53.744991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:53.744991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:53.744991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 314/636] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:53.745688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a067ca-2c3c-4cd8-a44f-c5dadaea5730 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:53.752847 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ce3cf-e6b0-4bbf-b9e0-fafb493dd5d7 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:53.790623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fd97d-035a-466e-8df9-df312c060d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:53.791301 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fd97d-035a-466e-8df9-df312c060d2c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:53.791583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fd97d-035a-466e-8df9-df312c060d2c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:53.791858 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fd97d-035a-466e-8df9-df312c060d2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:53.803088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7fd97d-035a-466e-8df9-df312c060d2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:53.805756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7fd97d-035a-466e-8df9-df312c060d2c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:53.812751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:53.820999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-5a7fd97d-035a-466e-8df9-df312c060d2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:53.820999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:53.820999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:53.820999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:53.820999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:53.820999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:53.820999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:53.820999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:53.820999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:53.820999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:53.820999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:53.820999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:53.820999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:53.820999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:53.820999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:53.822110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:53.823019 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:53.823019 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:53.823019 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:53.823019 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:53.823019 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 323/637] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:53.824209 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa9f88-d040-4373-b37c-49dffe904937 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:53.824437 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa9f88-d040-4373-b37c-49dffe904937 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:53.824678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa9f88-d040-4373-b37c-49dffe904937 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:53.830040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 324/638] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:53.830040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa9f88-d040-4373-b37c-49dffe904937 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:53.869863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a067ca-2c3c-4cd8-a44f-c5dadaea5730 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:53.878970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:53.915575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f035f-5d50-410d-92bc-3f97dbe9a096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:53.917321 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f035f-5d50-410d-92bc-3f97dbe9a096 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:53.917920 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f035f-5d50-410d-92bc-3f97dbe9a096 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:53.918200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f035f-5d50-410d-92bc-3f97dbe9a096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:53.933801 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f035f-5d50-410d-92bc-3f97dbe9a096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:53.936553 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f035f-5d50-410d-92bc-3f97dbe9a096 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:53.944975 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa9f88-d040-4373-b37c-49dffe904937 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:53.954270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-231f035f-5d50-410d-92bc-3f97dbe9a096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:53.954270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:53.954270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:53.954270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:53.954270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:53.954270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:53.954270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:53.954270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:53.954270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:53.954270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:53.954270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:53.954270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:53.954270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:53.954270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:53.954270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:53.955207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:53.956113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:53.956113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:53.956113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:53.956113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:53.956113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 315/639] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:53.956454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f035f-5d50-410d-92bc-3f97dbe9a096 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:53.989996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43201c-db0e-4fa8-a665-7c501dabc029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:53.990833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43201c-db0e-4fa8-a665-7c501dabc029 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:53.991112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43201c-db0e-4fa8-a665-7c501dabc029 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:53.991429 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43201c-db0e-4fa8-a665-7c501dabc029 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:53.992678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43201c-db0e-4fa8-a665-7c501dabc029 None admin] RBAC: Adding query filter params (name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:53.993607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43201c-db0e-4fa8-a665-7c501dabc029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:53.995103 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43201c-db0e-4fa8-a665-7c501dabc029 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:54.008978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 325/640] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:37:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:54.008978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c9d2d-d1c6-4b8a-b77d-9a40a717eb6d None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:54.008978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c9d2d-d1c6-4b8a-b77d-9a40a717eb6d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:54.008978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c9d2d-d1c6-4b8a-b77d-9a40a717eb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:54.024336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-635c9d2d-d1c6-4b8a-b77d-9a40a717eb6d None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:54.025595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-635c9d2d-d1c6-4b8a-b77d-9a40a717eb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:54.072100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f035f-5d50-410d-92bc-3f97dbe9a096 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:54.085785 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:54.100339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 326/641] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:54.100339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c9d2d-d1c6-4b8a-b77d-9a40a717eb6d None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:54.124038 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd24b0-934f-4054-8609-633c3810cfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:54.124852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd24b0-934f-4054-8609-633c3810cfd4 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:54.125138 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd24b0-934f-4054-8609-633c3810cfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:54.125824 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd24b0-934f-4054-8609-633c3810cfd4 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:54.126807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd24b0-934f-4054-8609-633c3810cfd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:54.127114 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd24b0-934f-4054-8609-633c3810cfd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:54.129150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd24b0-934f-4054-8609-633c3810cfd4 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:54.138041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:54.153241 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 316/642] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:54.153241 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd24b0-934f-4054-8609-633c3810cfd4 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:54.217957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c9d2d-d1c6-4b8a-b77d-9a40a717eb6d None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:54.260857 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c6f18-7ba3-4765-b8e9-3de3599aa35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:54.261590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c6f18-7ba3-4765-b8e9-3de3599aa35c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:54.261933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c6f18-7ba3-4765-b8e9-3de3599aa35c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:54.262278 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c6f18-7ba3-4765-b8e9-3de3599aa35c None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:54.263039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c6f18-7ba3-4765-b8e9-3de3599aa35c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:54.263410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c6f18-7ba3-4765-b8e9-3de3599aa35c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:54.264808 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c6f18-7ba3-4765-b8e9-3de3599aa35c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:54.268890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd24b0-934f-4054-8609-633c3810cfd4 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:54.283430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 327/643] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:54.283430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c6f18-7ba3-4765-b8e9-3de3599aa35c None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:54.306073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3d0b3-c8fc-4cfd-a257-45e7af2ea91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:54.307117 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d0b3-c8fc-4cfd-a257-45e7af2ea91a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:54.307117 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d0b3-c8fc-4cfd-a257-45e7af2ea91a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:54.307333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d0b3-c8fc-4cfd-a257-45e7af2ea91a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:54.317657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3d0b3-c8fc-4cfd-a257-45e7af2ea91a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:54.319061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3d0b3-c8fc-4cfd-a257-45e7af2ea91a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:54.334209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-59a3d0b3-c8fc-4cfd-a257-45e7af2ea91a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:54.334209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:54.334209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:54.334209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:54.334209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:54.334209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:54.334209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:54.334209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:54.334209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:54.334209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:54.334209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:54.334209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:54.334209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:54.334209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:54.334209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:54.335264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:54.336312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:54.336312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:54.336312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:54.336312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:54.336312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 317/644] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:54.336312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3d0b3-c8fc-4cfd-a257-45e7af2ea91a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:54.364600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:54.394484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:54.403041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c6f18-7ba3-4765-b8e9-3de3599aa35c None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:54.439678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b176be-8caa-4727-bb45-d19cc335d457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:54.440357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b176be-8caa-4727-bb45-d19cc335d457 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:54.440637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b176be-8caa-4727-bb45-d19cc335d457 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:54.440913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b176be-8caa-4727-bb45-d19cc335d457 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:54.452058 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b176be-8caa-4727-bb45-d19cc335d457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:54.452651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3d0b3-c8fc-4cfd-a257-45e7af2ea91a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-73b176be-8caa-4727-bb45-d19cc335d457 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:54.457449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:54.458445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:54.459426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:54.459426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:54.459426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:54.459426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:54.459426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:54.459426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:54.459426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:54.459426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:54.459426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:54.459426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:54.459426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:54.459426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:54.459426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 328/645] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:54.460192 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b176be-8caa-4727-bb45-d19cc335d457 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:54.494506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40fb42-894e-40e6-9c71-05ac8387915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:54.495308 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40fb42-894e-40e6-9c71-05ac8387915e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:54.495591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40fb42-894e-40e6-9c71-05ac8387915e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:54.495922 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40fb42-894e-40e6-9c71-05ac8387915e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:54.502770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40fb42-894e-40e6-9c71-05ac8387915e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:54.504493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40fb42-894e-40e6-9c71-05ac8387915e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:54.519037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-bb40fb42-894e-40e6-9c71-05ac8387915e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:54.519037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:54.519037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:54.519037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:54.519037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:54.519037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:54.519037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:54.519037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:54.519037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:54.519037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:54.519037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:54.519037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:54.519037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:37:54.519037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:37:54.519037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:54.520023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:54.520975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:37:54.520975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:37:54.520975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:37:54.520975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:37:54.520975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:37:54.520975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:37:54.520975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:54.520975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:54.520975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 318/646] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:54.521527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40fb42-894e-40e6-9c71-05ac8387915e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:54.619128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:54.639240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40fb42-894e-40e6-9c71-05ac8387915e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:54.678378 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf83231-a720-44ee-85a6-ec230b671790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:54.678920 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf83231-a720-44ee-85a6-ec230b671790 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:54.679202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf83231-a720-44ee-85a6-ec230b671790 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:54.679477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf83231-a720-44ee-85a6-ec230b671790 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:54.680465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf83231-a720-44ee-85a6-ec230b671790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:54.680773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf83231-a720-44ee-85a6-ec230b671790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:54.682148 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf83231-a720-44ee-85a6-ec230b671790 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:54.694245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b176be-8caa-4727-bb45-d19cc335d457 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:54.700154 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 319/647] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:54.700154 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf83231-a720-44ee-85a6-ec230b671790 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:54.735140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d66a9-ddf8-46bb-ac32-f0a5aefdcf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:54.735817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d66a9-ddf8-46bb-ac32-f0a5aefdcf27 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:54.736102 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d66a9-ddf8-46bb-ac32-f0a5aefdcf27 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:54.736419 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d66a9-ddf8-46bb-ac32-f0a5aefdcf27 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:54.743786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d66a9-ddf8-46bb-ac32-f0a5aefdcf27 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:54.745515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d66a9-ddf8-46bb-ac32-f0a5aefdcf27 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-570d66a9-ddf8-46bb-ac32-f0a5aefdcf27 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:54.757527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:54.758651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:54.759606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:54.759606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:54.759606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:54.759606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:54.759606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:54.759606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:54.759606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:54.759606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:54.759606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 03 18:37:54.759606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:54.759606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 329/648] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:54.760256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d66a9-ddf8-46bb-ac32-f0a5aefdcf27 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:54.776974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:54.810523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf83231-a720-44ee-85a6-ec230b671790 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:54.846179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27891091-e8f6-402b-8f5e-472f527f50b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:54.846805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27891091-e8f6-402b-8f5e-472f527f50b0 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:54.847087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27891091-e8f6-402b-8f5e-472f527f50b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:54.847359 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27891091-e8f6-402b-8f5e-472f527f50b0 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:54.848077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27891091-e8f6-402b-8f5e-472f527f50b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:54.848408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27891091-e8f6-402b-8f5e-472f527f50b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:54.849765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27891091-e8f6-402b-8f5e-472f527f50b0 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:54.859869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 320/649] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:37:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:54.860566 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27891091-e8f6-402b-8f5e-472f527f50b0 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:54.867881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d66a9-ddf8-46bb-ac32-f0a5aefdcf27 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:54.870591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:54.906861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb0429-21a2-45ac-882d-c37d6b4f0fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:54.907565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb0429-21a2-45ac-882d-c37d6b4f0fc6 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:54.907751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb0429-21a2-45ac-882d-c37d6b4f0fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:54.908012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb0429-21a2-45ac-882d-c37d6b4f0fc6 None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:54.908846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb0429-21a2-45ac-882d-c37d6b4f0fc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:54.909150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb0429-21a2-45ac-882d-c37d6b4f0fc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:54.910617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb0429-21a2-45ac-882d-c37d6b4f0fc6 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:54.926594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 330/650] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:37:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:54.927438 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb0429-21a2-45ac-882d-c37d6b4f0fc6 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:54.981577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27891091-e8f6-402b-8f5e-472f527f50b0 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:55.014372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:55.017399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f554c-a812-46a2-8daa-62c976c80d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:55.018035 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f554c-a812-46a2-8daa-62c976c80d04 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:55.018267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f554c-a812-46a2-8daa-62c976c80d04 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:55.018555 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f554c-a812-46a2-8daa-62c976c80d04 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:55.026901 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f554c-a812-46a2-8daa-62c976c80d04 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:55.028400 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f554c-a812-46a2-8daa-62c976c80d04 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-ac0f554c-a812-46a2-8daa-62c976c80d04 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:55.039890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:55.041008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:55.042135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:55.042135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:55.042135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:55.042135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:55.042135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:55.042135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:55.042135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:55.042135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:55.042135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 18:37:55.042135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:55.045826 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 321/651] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:55.046665 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f554c-a812-46a2-8daa-62c976c80d04 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:55.047485 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb0429-21a2-45ac-882d-c37d6b4f0fc6 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:55.086596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14141c-8727-4e9e-a725-41a55df58f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:55.087273 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14141c-8727-4e9e-a725-41a55df58f4b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:55.087554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14141c-8727-4e9e-a725-41a55df58f4b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:55.087826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14141c-8727-4e9e-a725-41a55df58f4b None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:55.088894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14141c-8727-4e9e-a725-41a55df58f4b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:55.089025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14141c-8727-4e9e-a725-41a55df58f4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:55.090404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14141c-8727-4e9e-a725-41a55df58f4b None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:55.110196 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 331/652] 10.222.0.69 () {56 vars in 1137 bytes} [Wed Aug 3 18:37:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:55.111354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14141c-8727-4e9e-a725-41a55df58f4b None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:55.123546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:55.162557 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f554c-a812-46a2-8daa-62c976c80d04 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:55.201246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93def30-2bbd-4380-96b0-227e6a94b90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:55.201978 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93def30-2bbd-4380-96b0-227e6a94b90c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:55.202265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93def30-2bbd-4380-96b0-227e6a94b90c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:55.202540 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93def30-2bbd-4380-96b0-227e6a94b90c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:55.214227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93def30-2bbd-4380-96b0-227e6a94b90c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:55.215685 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93def30-2bbd-4380-96b0-227e6a94b90c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:55.223483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14141c-8727-4e9e-a725-41a55df58f4b None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:55.231215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-d93def30-2bbd-4380-96b0-227e6a94b90c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:55.231215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:55.231215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:55.231215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:55.231215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:55.231215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:55.231215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:55.231215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:55.231215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:55.231215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:55.231215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:55.231215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:55.231215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:55.231215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:55.231215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:55.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:55.233482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:55.233482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:55.233482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:55.233482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:55.233818 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 322/653] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:55.234686 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93def30-2bbd-4380-96b0-227e6a94b90c None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:55.263787 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:55.270067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9907e80-27cb-41dc-9371-3b6c9def0a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:55.270690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9907e80-27cb-41dc-9371-3b6c9def0a99 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:55.271023 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9907e80-27cb-41dc-9371-3b6c9def0a99 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:55.271303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9907e80-27cb-41dc-9371-3b6c9def0a99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:55.272107 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9907e80-27cb-41dc-9371-3b6c9def0a99 None admin] RBAC: Adding query filter params (role.id=d67cdb8bae3648fd8f3589ab33176d14, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=850a50cfe4394febbafb0908b08888b2) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:55.272451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9907e80-27cb-41dc-9371-3b6c9def0a99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:55.274692 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9907e80-27cb-41dc-9371-3b6c9def0a99 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:55.289604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 332/654] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:55 2022] GET /identity/v3/role_assignments?user.id=850a50cfe4394febbafb0908b08888b2&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=d67cdb8bae3648fd8f3589ab33176d14 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:55.289604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9907e80-27cb-41dc-9371-3b6c9def0a99 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:55.366794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93def30-2bbd-4380-96b0-227e6a94b90c None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:55.387883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:55.407753 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9907e80-27cb-41dc-9371-3b6c9def0a99 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:55.411191 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b8a07-7cc8-4a9e-b07a-be85372df391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:55.412117 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b8a07-7cc8-4a9e-b07a-be85372df391 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:55.412648 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b8a07-7cc8-4a9e-b07a-be85372df391 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:55.413126 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b8a07-7cc8-4a9e-b07a-be85372df391 None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:55.414744 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b8a07-7cc8-4a9e-b07a-be85372df391 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:55.415358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b8a07-7cc8-4a9e-b07a-be85372df391 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:55.417219 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b8a07-7cc8-4a9e-b07a-be85372df391 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:55.436853 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 323/655] 10.222.0.69 () {56 vars in 1137 bytes} [Wed Aug 3 18:37:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:55.437701 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b8a07-7cc8-4a9e-b07a-be85372df391 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:55.455286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10299fa0-8589-4cbc-af71-7d27fedec5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:55.455952 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10299fa0-8589-4cbc-af71-7d27fedec5a8 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:55.456235 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10299fa0-8589-4cbc-af71-7d27fedec5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:55.456540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10299fa0-8589-4cbc-af71-7d27fedec5a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:55.528892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:55.537999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10299fa0-8589-4cbc-af71-7d27fedec5a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:55.540456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10299fa0-8589-4cbc-af71-7d27fedec5a8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:55.563530 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b8a07-7cc8-4a9e-b07a-be85372df391 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:55.599615 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af49d42-f514-4b66-bf20-17edf94d7510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:55.600395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af49d42-f514-4b66-bf20-17edf94d7510 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:55.600683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af49d42-f514-4b66-bf20-17edf94d7510 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:55.600959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af49d42-f514-4b66-bf20-17edf94d7510 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:55.602274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af49d42-f514-4b66-bf20-17edf94d7510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:55.602800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af49d42-f514-4b66-bf20-17edf94d7510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:55.604248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af49d42-f514-4b66-bf20-17edf94d7510 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:55.615873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 324/656] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:55.621433 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08653d-1a97-4003-af16-56cc9d00e53d None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:55.621722 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08653d-1a97-4003-af16-56cc9d00e53d None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:55.622004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08653d-1a97-4003-af16-56cc9d00e53d None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:55.623432 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 325/657] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:55.624004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08653d-1a97-4003-af16-56cc9d00e53d None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:55.654930 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 333/658] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:55.655885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10299fa0-8589-4cbc-af71-7d27fedec5a8 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:55.665233 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:55.740763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08653d-1a97-4003-af16-56cc9d00e53d None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:55.756489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:55.764253 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10299fa0-8589-4cbc-af71-7d27fedec5a8 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:55.781728 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081d49d-b692-459e-a0bd-6573856fe825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:55.782718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081d49d-b692-459e-a0bd-6573856fe825 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:55.783171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081d49d-b692-459e-a0bd-6573856fe825 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:55.783596 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081d49d-b692-459e-a0bd-6573856fe825 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:55.802804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e7e89-23b1-4153-92b8-6747f9952081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:55.803449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e7e89-23b1-4153-92b8-6747f9952081 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:55.803733 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e7e89-23b1-4153-92b8-6747f9952081 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:55.804006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e7e89-23b1-4153-92b8-6747f9952081 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:55.813110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e7e89-23b1-4153-92b8-6747f9952081 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:55.814722 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e7e89-23b1-4153-92b8-6747f9952081 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-ca3e7e89-23b1-4153-92b8-6747f9952081 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:55.828561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:55.829648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:55.830604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:55.830604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:55.830604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:55.830604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:55.830604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:55.830604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:55.830604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:55.830604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:55.830604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 18:37:55.830604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:55.830604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 334/659] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:55.831231 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b20ef6-ddbb-43fe-a711-7e1ce0107630 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:55.831777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b20ef6-ddbb-43fe-a711-7e1ce0107630 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:55.831777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b20ef6-ddbb-43fe-a711-7e1ce0107630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:55.850527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d8b20ef6-ddbb-43fe-a711-7e1ce0107630 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:55.851836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d8b20ef6-ddbb-43fe-a711-7e1ce0107630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:55.866042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081d49d-b692-459e-a0bd-6573856fe825 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:55.867800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081d49d-b692-459e-a0bd-6573856fe825 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:55.903873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:55.935645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 335/660] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:55.935645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b20ef6-ddbb-43fe-a711-7e1ce0107630 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:55.991864 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 326/661] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:55.993084 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081d49d-b692-459e-a0bd-6573856fe825 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:56.027554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:56.044231 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b20ef6-ddbb-43fe-a711-7e1ce0107630 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:56.082204 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc916eb1-c33d-4c18-ac1f-6c07da015ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:56.082842 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc916eb1-c33d-4c18-ac1f-6c07da015ecd None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:56.083133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc916eb1-c33d-4c18-ac1f-6c07da015ecd None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:56.083410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc916eb1-c33d-4c18-ac1f-6c07da015ecd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:56.095494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc916eb1-c33d-4c18-ac1f-6c07da015ecd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:56.096980 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc916eb1-c33d-4c18-ac1f-6c07da015ecd None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:56.104309 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081d49d-b692-459e-a0bd-6573856fe825 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:56.112635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-cc916eb1-c33d-4c18-ac1f-6c07da015ecd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:56.112635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:56.112635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:56.112635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:56.112635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:56.112635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:56.112635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:56.112635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:56.112635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:56.112635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:56.112635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:56.112635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:56.112635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:56.112635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:56.112635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:56.113553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:56.114459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:56.114459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:56.114459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:56.114459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:56.114459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 336/662] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:56.114747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc916eb1-c33d-4c18-ac1f-6c07da015ecd None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:56.146487 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d7bbb-dde8-4d79-939a-7ec45275036d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:56.147108 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d7bbb-dde8-4d79-939a-7ec45275036d None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:56.147388 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d7bbb-dde8-4d79-939a-7ec45275036d None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:56.147657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d7bbb-dde8-4d79-939a-7ec45275036d None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:56.149231 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d7bbb-dde8-4d79-939a-7ec45275036d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:56.149599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d7bbb-dde8-4d79-939a-7ec45275036d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:56.151047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d7bbb-dde8-4d79-939a-7ec45275036d None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:56.154454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:56.170482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 327/663] 10.222.0.69 () {56 vars in 1139 bytes} [Wed Aug 3 18:37:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:56.175644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d7bbb-dde8-4d79-939a-7ec45275036d None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:56.224673 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc916eb1-c33d-4c18-ac1f-6c07da015ecd None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:56.262190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11025008-6a95-4b80-b6a7-0c3d8dfc3e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:56.268373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11025008-6a95-4b80-b6a7-0c3d8dfc3e53 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:56.268373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11025008-6a95-4b80-b6a7-0c3d8dfc3e53 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:56.268373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11025008-6a95-4b80-b6a7-0c3d8dfc3e53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:56.275304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:56.279350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11025008-6a95-4b80-b6a7-0c3d8dfc3e53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-11025008-6a95-4b80-b6a7-0c3d8dfc3e53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:56.284472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:56.285701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:56.286607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:56.286607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:56.286607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:56.286607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:56.286607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:56.286607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:56.286607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:56.286607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:56.286607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:56.286607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:56.286607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:37:56.286607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:56.286607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 337/664] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:37:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:56.287370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11025008-6a95-4b80-b6a7-0c3d8dfc3e53 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:56.298075 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d7bbb-dde8-4d79-939a-7ec45275036d None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:56.336754 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8d395-8032-434e-9e96-d21114f1a812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:56.337400 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8d395-8032-434e-9e96-d21114f1a812 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:56.337677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8d395-8032-434e-9e96-d21114f1a812 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:56.337948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8d395-8032-434e-9e96-d21114f1a812 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:56.348851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8d395-8032-434e-9e96-d21114f1a812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:56.350284 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8d395-8032-434e-9e96-d21114f1a812 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:56.366052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-c2b8d395-8032-434e-9e96-d21114f1a812 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:56.366052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:56.366052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:56.366052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:56.366052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:56.366052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:56.366052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:56.366052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:56.366052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:56.366052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:56.366052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:56.366052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:56.366052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:56.366052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:56.366052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:56.367571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:56.368595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:56.368595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:56.368595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:56.368595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:56.372556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 328/665] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:56.373434 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8d395-8032-434e-9e96-d21114f1a812 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:56.405946 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11025008-6a95-4b80-b6a7-0c3d8dfc3e53 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:56.429542 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:56.446069 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa04471-4713-47bd-897f-505d87154ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:56.446696 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa04471-4713-47bd-897f-505d87154ed3 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:56.447023 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa04471-4713-47bd-897f-505d87154ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:56.447303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa04471-4713-47bd-897f-505d87154ed3 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:56.448008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa04471-4713-47bd-897f-505d87154ed3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:56.448331 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa04471-4713-47bd-897f-505d87154ed3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:56.449738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa04471-4713-47bd-897f-505d87154ed3 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:56.463820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 338/666] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:56.464819 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa04471-4713-47bd-897f-505d87154ed3 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:56.499586 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8d395-8032-434e-9e96-d21114f1a812 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:56.534772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:56.539322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c4fc4-4c22-42ed-98cb-0353a68517aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:56.539956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c4fc4-4c22-42ed-98cb-0353a68517aa None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:56.540270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c4fc4-4c22-42ed-98cb-0353a68517aa None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:56.540571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c4fc4-4c22-42ed-98cb-0353a68517aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:56.579110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa04471-4713-47bd-897f-505d87154ed3 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:56.614622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c4fc4-4c22-42ed-98cb-0353a68517aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:56.616172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86f32f-e7da-48a0-a0da-4d8fdba82199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:56.616472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c4fc4-4c22-42ed-98cb-0353a68517aa None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:56.616838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86f32f-e7da-48a0-a0da-4d8fdba82199 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:56.617275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86f32f-e7da-48a0-a0da-4d8fdba82199 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:56.617580 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86f32f-e7da-48a0-a0da-4d8fdba82199 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:56.618567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86f32f-e7da-48a0-a0da-4d8fdba82199 None admin] RBAC: Adding query filter params (name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:56.618782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86f32f-e7da-48a0-a0da-4d8fdba82199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:56.620992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86f32f-e7da-48a0-a0da-4d8fdba82199 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:56.635951 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 339/667] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:37:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:56.637215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86f32f-e7da-48a0-a0da-4d8fdba82199 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:56.665359 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:56.723648 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 329/668] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:56.725008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c4fc4-4c22-42ed-98cb-0353a68517aa None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:56.748194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86f32f-e7da-48a0-a0da-4d8fdba82199 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:56.770927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:56.786745 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5f588-4835-4c52-b72c-43dd9b9cf814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:56.787283 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5f588-4835-4c52-b72c-43dd9b9cf814 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:56.787569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5f588-4835-4c52-b72c-43dd9b9cf814 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:56.787847 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5f588-4835-4c52-b72c-43dd9b9cf814 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:56.788596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5f588-4835-4c52-b72c-43dd9b9cf814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:56.789531 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5f588-4835-4c52-b72c-43dd9b9cf814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:56.791099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5f588-4835-4c52-b72c-43dd9b9cf814 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:56.808614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 340/669] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:56.808614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5f588-4835-4c52-b72c-43dd9b9cf814 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:56.847058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c4fc4-4c22-42ed-98cb-0353a68517aa None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:56.885329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984122e-6419-4603-b6b9-1547d03b837f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:56.885968 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984122e-6419-4603-b6b9-1547d03b837f None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:56.886252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984122e-6419-4603-b6b9-1547d03b837f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:56.886528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984122e-6419-4603-b6b9-1547d03b837f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:56.894197 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984122e-6419-4603-b6b9-1547d03b837f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:56.895744 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984122e-6419-4603-b6b9-1547d03b837f None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:56.908949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-2984122e-6419-4603-b6b9-1547d03b837f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:56.908949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:56.908949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:56.908949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:56.908949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:56.908949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:56.908949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:56.908949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:56.908949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:56.908949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:56.908949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:56.908949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:56.908949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:37:56.908949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:37:56.908949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:56.909931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:56.910848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:37:56.910848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:37:56.910848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:37:56.910848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:37:56.910848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:37:56.910848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:37:56.910848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:56.910848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:56.910848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 330/670] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:56.911402 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984122e-6419-4603-b6b9-1547d03b837f None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:56.922936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:56.930216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5f588-4835-4c52-b72c-43dd9b9cf814 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:56.970541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20fb57-6b8e-4ce8-999c-ac8172c2366a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:56.971181 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20fb57-6b8e-4ce8-999c-ac8172c2366a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:56.971514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20fb57-6b8e-4ce8-999c-ac8172c2366a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:56.971794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20fb57-6b8e-4ce8-999c-ac8172c2366a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:56.984453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20fb57-6b8e-4ce8-999c-ac8172c2366a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:56.986074 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20fb57-6b8e-4ce8-999c-ac8172c2366a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:57.002258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-3c20fb57-6b8e-4ce8-999c-ac8172c2366a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:57.002258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:57.002258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:57.002258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:57.002258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:57.002258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:57.002258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:57.002258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:57.002258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:57.002258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:57.002258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:57.002258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:57.002258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:57.002258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:57.002258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:57.003525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:57.004915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:57.004915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:57.004915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:57.004915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:57.009018 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 341/671] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:57.010066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20fb57-6b8e-4ce8-999c-ac8172c2366a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:57.034886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984122e-6419-4603-b6b9-1547d03b837f None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:57.041593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:57.074813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588a6bb-6b9b-4dbf-bc5c-4c805e109461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:57.075467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588a6bb-6b9b-4dbf-bc5c-4c805e109461 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:57.075751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588a6bb-6b9b-4dbf-bc5c-4c805e109461 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:57.076026 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588a6bb-6b9b-4dbf-bc5c-4c805e109461 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:57.115640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20fb57-6b8e-4ce8-999c-ac8172c2366a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:57.153408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4b1c9-455b-4aac-a152-c75e845994a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:57.154075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4b1c9-455b-4aac-a152-c75e845994a7 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:57.154361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4b1c9-455b-4aac-a152-c75e845994a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:57.154638 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4b1c9-455b-4aac-a152-c75e845994a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:57.162303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad4b1c9-455b-4aac-a152-c75e845994a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:57.163823 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad4b1c9-455b-4aac-a152-c75e845994a7 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:57.176982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-3ad4b1c9-455b-4aac-a152-c75e845994a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:57.176982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:57.176982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:57.176982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:57.176982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:57.176982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:57.176982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:57.176982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:57.176982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:57.176982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:57.176982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:57.176982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:57.176982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:37:57.176982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:37:57.176982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:57.177962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:57.178868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:37:57.178868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:37:57.178868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:37:57.178868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:37:57.178868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:37:57.178868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:37:57.178868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:57.178868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:57.178868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 342/672] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:57.180641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4b1c9-455b-4aac-a152-c75e845994a7 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:57.213903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588a6bb-6b9b-4dbf-bc5c-4c805e109461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:57.215389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588a6bb-6b9b-4dbf-bc5c-4c805e109461 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:57.232468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-f588a6bb-6b9b-4dbf-bc5c-4c805e109461 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:57.232468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:57.232468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:57.232468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:57.232468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:57.232468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:57.232468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:57.232468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:57.232468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:57.232468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:57.232468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:57.232468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:57.232468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:57.232468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:57.232468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:57.233449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:57.234379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:57.234379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:57.234379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:57.234379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:57.238326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 331/673] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:57.238889 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588a6bb-6b9b-4dbf-bc5c-4c805e109461 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:57.292072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:57.301765 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4b1c9-455b-4aac-a152-c75e845994a7 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:57.307155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:57.340058 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab804c-2319-4b98-8714-19ae1e59c96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:57.340717 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab804c-2319-4b98-8714-19ae1e59c96b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:57.341013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab804c-2319-4b98-8714-19ae1e59c96b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:57.341269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab804c-2319-4b98-8714-19ae1e59c96b None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:57.342032 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab804c-2319-4b98-8714-19ae1e59c96b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:57.342341 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab804c-2319-4b98-8714-19ae1e59c96b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:57.343779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab804c-2319-4b98-8714-19ae1e59c96b None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:57.350615 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588a6bb-6b9b-4dbf-bc5c-4c805e109461 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:57.362265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 343/674] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:37:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:57.362265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab804c-2319-4b98-8714-19ae1e59c96b None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:57.393527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884c7f8-3ecb-4424-9d03-749a1817b69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:57.394308 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884c7f8-3ecb-4424-9d03-749a1817b69f None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:57.394594 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884c7f8-3ecb-4424-9d03-749a1817b69f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:57.395206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884c7f8-3ecb-4424-9d03-749a1817b69f None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:57.396181 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884c7f8-3ecb-4424-9d03-749a1817b69f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:57.396522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884c7f8-3ecb-4424-9d03-749a1817b69f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:57.398046 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884c7f8-3ecb-4424-9d03-749a1817b69f None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:57.416607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 332/675] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:57.416607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884c7f8-3ecb-4424-9d03-749a1817b69f None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:57.464814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab804c-2319-4b98-8714-19ae1e59c96b None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:57.501018 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48f516-4a41-470d-a471-b52045de8730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:57.501909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48f516-4a41-470d-a471-b52045de8730 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:57.502187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48f516-4a41-470d-a471-b52045de8730 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:57.502512 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48f516-4a41-470d-a471-b52045de8730 None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:57.503525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48f516-4a41-470d-a471-b52045de8730 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:57.503831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48f516-4a41-470d-a471-b52045de8730 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:57.505317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48f516-4a41-470d-a471-b52045de8730 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:57.516950 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 344/676] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:37:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:57.517324 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884c7f8-3ecb-4424-9d03-749a1817b69f None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:57.517757 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48f516-4a41-470d-a471-b52045de8730 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:57.535287 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:57.535460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:57.554405 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3515d0e-4c84-43f8-8b54-0b403c449f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:57.555750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3515d0e-4c84-43f8-8b54-0b403c449f16 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:57.556096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3515d0e-4c84-43f8-8b54-0b403c449f16 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:57.556490 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3515d0e-4c84-43f8-8b54-0b403c449f16 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:57.557198 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3515d0e-4c84-43f8-8b54-0b403c449f16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:57.557504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3515d0e-4c84-43f8-8b54-0b403c449f16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:57.558884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3515d0e-4c84-43f8-8b54-0b403c449f16 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:57.570333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 333/677] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:57.571511 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348cf34-bb7f-4608-a7fa-23f04c74b6da None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:57.571806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348cf34-bb7f-4608-a7fa-23f04c74b6da None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:57.572307 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348cf34-bb7f-4608-a7fa-23f04c74b6da None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:57.573510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 334/678] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:37:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:37:57.574069 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348cf34-bb7f-4608-a7fa-23f04c74b6da None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:57.626334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48f516-4a41-470d-a471-b52045de8730 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:57.662634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a31a8-b29c-4ffe-af07-fadcafa04c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:57.663245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a31a8-b29c-4ffe-af07-fadcafa04c38 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:57.663522 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a31a8-b29c-4ffe-af07-fadcafa04c38 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:57.664407 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a31a8-b29c-4ffe-af07-fadcafa04c38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:57.665516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a31a8-b29c-4ffe-af07-fadcafa04c38 None admin] RBAC: Adding query filter params (role.id=d67cdb8bae3648fd8f3589ab33176d14, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=4799b0c9e5aa47d389121d2183666d3d) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:57.665883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a31a8-b29c-4ffe-af07-fadcafa04c38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:57.667304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a31a8-b29c-4ffe-af07-fadcafa04c38 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:57.673165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 345/679] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:57 2022] GET /identity/v3/role_assignments?user.id=4799b0c9e5aa47d389121d2183666d3d&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=d67cdb8bae3648fd8f3589ab33176d14 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:57.673933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a31a8-b29c-4ffe-af07-fadcafa04c38 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:57.680002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348cf34-bb7f-4608-a7fa-23f04c74b6da None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:57.725091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3875876-6cd2-4411-908a-23739cb9a192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:57.727465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3875876-6cd2-4411-908a-23739cb9a192 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:57.727853 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3875876-6cd2-4411-908a-23739cb9a192 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:57.728129 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3875876-6cd2-4411-908a-23739cb9a192 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:57.736897 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3875876-6cd2-4411-908a-23739cb9a192 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:57.738465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3875876-6cd2-4411-908a-23739cb9a192 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-e3875876-6cd2-4411-908a-23739cb9a192 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:57.751513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:57.752591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:57.753842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:57.753842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:57.753842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:57.753842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:57.753842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:57.753842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:57.753842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:57.753842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:57.753842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 18:37:57.753842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:57.753842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 335/680] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:37:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:57.753842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3875876-6cd2-4411-908a-23739cb9a192 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:57.783399 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:57.803424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:57.812244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a31a8-b29c-4ffe-af07-fadcafa04c38 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:57.855652 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b02b77-aadf-4d94-9f8e-04218621b9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:57.856613 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b02b77-aadf-4d94-9f8e-04218621b9a4 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:57.856898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b02b77-aadf-4d94-9f8e-04218621b9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:57.857176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b02b77-aadf-4d94-9f8e-04218621b9a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:57.857978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b02b77-aadf-4d94-9f8e-04218621b9a4 None admin] RBAC: Adding query filter params (role.id=d67cdb8bae3648fd8f3589ab33176d14, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=c7d0fde945184c93b8fd91d15b66f534) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:57.858295 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b02b77-aadf-4d94-9f8e-04218621b9a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:57.859800 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b02b77-aadf-4d94-9f8e-04218621b9a4 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:57.867008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 346/681] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:57 2022] GET /identity/v3/role_assignments?user.id=c7d0fde945184c93b8fd91d15b66f534&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=d67cdb8bae3648fd8f3589ab33176d14 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:57.868237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47558c-e823-44fb-8b8c-a7c77c258eab None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:57.868556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47558c-e823-44fb-8b8c-a7c77c258eab None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:57.868832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47558c-e823-44fb-8b8c-a7c77c258eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:57.878701 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3875876-6cd2-4411-908a-23739cb9a192 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:57.893309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-8f47558c-e823-44fb-8b8c-a7c77c258eab None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:37:57.894636 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8f47558c-e823-44fb-8b8c-a7c77c258eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:37:57.933439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f05c3-d94c-4f53-b4db-55b6685f5792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:57.935476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f05c3-d94c-4f53-b4db-55b6685f5792 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:57.936025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f05c3-d94c-4f53-b4db-55b6685f5792 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:57.936622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f05c3-d94c-4f53-b4db-55b6685f5792 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:57.947994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f05c3-d94c-4f53-b4db-55b6685f5792 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:57.949809 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f05c3-d94c-4f53-b4db-55b6685f5792 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:57.967170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-146f05c3-d94c-4f53-b4db-55b6685f5792 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:57.967170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:57.967170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:57.967170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:57.967170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:57.967170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:57.967170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:57.967170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:57.967170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:57.967170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:57.967170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:57.967170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:57.967170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:37:57.967170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:37:57.967170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:57.968155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:57.969112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:37:57.969112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:37:57.969112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:37:57.969112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:37:57.969112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:37:57.969112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:37:57.969112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:57.969112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:57.969112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 336/682] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:57.969112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f05c3-d94c-4f53-b4db-55b6685f5792 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:57.999514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 347/683] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:37:58.001897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47558c-e823-44fb-8b8c-a7c77c258eab None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:58.089173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:58.095947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f05c3-d94c-4f53-b4db-55b6685f5792 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:58.099961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:58.138356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47558c-e823-44fb-8b8c-a7c77c258eab None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:58.142937 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a79121-eeb2-4cce-bba2-363b0307ee3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:58.143750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a79121-eeb2-4cce-bba2-363b0307ee3a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:58.144186 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a79121-eeb2-4cce-bba2-363b0307ee3a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:58.144631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a79121-eeb2-4cce-bba2-363b0307ee3a None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:58.145608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a79121-eeb2-4cce-bba2-363b0307ee3a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:58.146330 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a79121-eeb2-4cce-bba2-363b0307ee3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:58.147962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a79121-eeb2-4cce-bba2-363b0307ee3a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:58.168801 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 337/684] 10.222.0.69 () {56 vars in 1135 bytes} [Wed Aug 3 18:37:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:58.169721 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a79121-eeb2-4cce-bba2-363b0307ee3a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:58.177074 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bedb17-d701-453d-8a2a-79be07b768bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:58.177737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bedb17-d701-453d-8a2a-79be07b768bb None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:58.178022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bedb17-d701-453d-8a2a-79be07b768bb None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:58.178298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bedb17-d701-453d-8a2a-79be07b768bb None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:58.179062 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bedb17-d701-453d-8a2a-79be07b768bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:58.179774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bedb17-d701-453d-8a2a-79be07b768bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:58.181398 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bedb17-d701-453d-8a2a-79be07b768bb None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:58.194233 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 348/685] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:37:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:58.195849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bedb17-d701-453d-8a2a-79be07b768bb None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:58.288665 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a79121-eeb2-4cce-bba2-363b0307ee3a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:58.303364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bedb17-d701-453d-8a2a-79be07b768bb None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:58.334492 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:58.338173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f2f6b-c89a-4120-a6dd-e8bbdfe3ab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:58.339298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f2f6b-c89a-4120-a6dd-e8bbdfe3ab87 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:58.339491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f2f6b-c89a-4120-a6dd-e8bbdfe3ab87 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:58.339771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f2f6b-c89a-4120-a6dd-e8bbdfe3ab87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:58.346476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94865e3-8487-4c9c-b190-a7595f540e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:58.347346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94865e3-8487-4c9c-b190-a7595f540e51 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:58.347828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94865e3-8487-4c9c-b190-a7595f540e51 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:58.348309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94865e3-8487-4c9c-b190-a7595f540e51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:58.351561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f2f6b-c89a-4120-a6dd-e8bbdfe3ab87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-804f2f6b-c89a-4120-a6dd-e8bbdfe3ab87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:37:58.356572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:58.357572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:58.358467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:58.358467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:58.358467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:58.358467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:58.358467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:37:58.358467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:37:58.358467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:37:58.358467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:37:58.358467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:37:58.358467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:37:58.358467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:37:58.358467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:58.362531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 338/686] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:37:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:58.363571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f2f6b-c89a-4120-a6dd-e8bbdfe3ab87 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:58.385817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:58.428028 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94865e3-8487-4c9c-b190-a7595f540e51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:58.430313 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94865e3-8487-4c9c-b190-a7595f540e51 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:58.481992 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f2f6b-c89a-4120-a6dd-e8bbdfe3ab87 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:58.521620 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7295b3b-1e75-4299-9afb-06e1b039338a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:58.522627 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7295b3b-1e75-4299-9afb-06e1b039338a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:58.523114 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7295b3b-1e75-4299-9afb-06e1b039338a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:58.523571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7295b3b-1e75-4299-9afb-06e1b039338a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:58.524623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7295b3b-1e75-4299-9afb-06e1b039338a None admin] RBAC: Adding query filter params (role.id=d67cdb8bae3648fd8f3589ab33176d14, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=0eb9ad672e5444098d71b7c9a27b72e8) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:58.525081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7295b3b-1e75-4299-9afb-06e1b039338a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:58.526750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7295b3b-1e75-4299-9afb-06e1b039338a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:58.534817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 339/687] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:58 2022] GET /identity/v3/role_assignments?user.id=0eb9ad672e5444098d71b7c9a27b72e8&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=d67cdb8bae3648fd8f3589ab33176d14 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:58.541471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 349/688] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:37:58.541696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7295b3b-1e75-4299-9afb-06e1b039338a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:58.545796 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94865e3-8487-4c9c-b190-a7595f540e51 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:58.602897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:58.723612 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:58.740670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94865e3-8487-4c9c-b190-a7595f540e51 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:58.796277 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7295b3b-1e75-4299-9afb-06e1b039338a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:58.818365 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98f750-8af1-41ba-9e1b-ecd50a18e4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:58.819829 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98f750-8af1-41ba-9e1b-ecd50a18e4c0 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:58.821358 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98f750-8af1-41ba-9e1b-ecd50a18e4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:58.822286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98f750-8af1-41ba-9e1b-ecd50a18e4c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:58.832854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc6807-845a-40de-ab7f-7264f669c95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:58.833481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc6807-845a-40de-ab7f-7264f669c95f None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:58.833763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc6807-845a-40de-ab7f-7264f669c95f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:58.834037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc6807-845a-40de-ab7f-7264f669c95f None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:58.834762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc6807-845a-40de-ab7f-7264f669c95f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:58.835067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc6807-845a-40de-ab7f-7264f669c95f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:58.837221 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc6807-845a-40de-ab7f-7264f669c95f None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:58.846207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 340/689] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:37:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:58.849820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98f750-8af1-41ba-9e1b-ecd50a18e4c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:58.852102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc6807-845a-40de-ab7f-7264f669c95f None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:58.853528 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98f750-8af1-41ba-9e1b-ecd50a18e4c0 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:58.883351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-1b98f750-8af1-41ba-9e1b-ecd50a18e4c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:58.883351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:58.883351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:58.883351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:58.883351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:58.883351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:58.883351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:58.883351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:58.883351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:58.883351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:58.883351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:58.883351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:58.883351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:58.883351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:58.883351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:58.885378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:58.887482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:58.887482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:58.887482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:58.887482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:58.890557 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 350/690] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:37:58.899950 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98f750-8af1-41ba-9e1b-ecd50a18e4c0 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:58.972531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc6807-845a-40de-ab7f-7264f669c95f None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:59.007198 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa92113-16d5-45de-af8a-91d590f38a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:59.008558 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa92113-16d5-45de-af8a-91d590f38a51 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:59.008845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa92113-16d5-45de-af8a-91d590f38a51 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:59.009122 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa92113-16d5-45de-af8a-91d590f38a51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:59.020622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa92113-16d5-45de-af8a-91d590f38a51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:59.022063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa92113-16d5-45de-af8a-91d590f38a51 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:59.024663 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:59.040161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-bfa92113-16d5-45de-af8a-91d590f38a51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:59.040161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:59.040161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:59.040161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:59.040161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:59.040161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:59.040161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:59.040161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:59.040161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:59.040161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:59.040161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:59.040161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:59.040161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:37:59.040161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:37:59.040161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:59.042178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:59.044208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:37:59.044208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:37:59.044208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:37:59.044208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:59.044208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 341/691] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:59.047200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa92113-16d5-45de-af8a-91d590f38a51 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:59.064579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:59.102352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98f750-8af1-41ba-9e1b-ecd50a18e4c0 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:59.159875 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa92113-16d5-45de-af8a-91d590f38a51 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:59.168277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143739f5-f3f9-495e-adcf-14abf74a3cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:59.169571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143739f5-f3f9-495e-adcf-14abf74a3cbf None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:59.170288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143739f5-f3f9-495e-adcf-14abf74a3cbf None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:59.171028 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143739f5-f3f9-495e-adcf-14abf74a3cbf None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:59.172455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143739f5-f3f9-495e-adcf-14abf74a3cbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:59.173195 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143739f5-f3f9-495e-adcf-14abf74a3cbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:59.175920 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143739f5-f3f9-495e-adcf-14abf74a3cbf None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:59.196532 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b8596-4881-4d49-8322-9c5a2d8ff352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:59.197157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b8596-4881-4d49-8322-9c5a2d8ff352 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:59.197441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b8596-4881-4d49-8322-9c5a2d8ff352 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:59.197718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b8596-4881-4d49-8322-9c5a2d8ff352 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:59.198429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b8596-4881-4d49-8322-9c5a2d8ff352 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:59.198733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b8596-4881-4d49-8322-9c5a2d8ff352 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:59.200104 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b8596-4881-4d49-8322-9c5a2d8ff352 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:59.201668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 351/692] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:37:59.210040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143739f5-f3f9-495e-adcf-14abf74a3cbf None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:59.212991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 342/693] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:59.218994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b8596-4881-4d49-8322-9c5a2d8ff352 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:59.277984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:59.329377 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b8596-4881-4d49-8322-9c5a2d8ff352 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:59.365269 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b535f3-2f4d-479f-80e7-22fd5a69c0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:59.365942 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b535f3-2f4d-479f-80e7-22fd5a69c0cc None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:59.366248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b535f3-2f4d-479f-80e7-22fd5a69c0cc None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:59.366470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b535f3-2f4d-479f-80e7-22fd5a69c0cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:59.375243 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b535f3-2f4d-479f-80e7-22fd5a69c0cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:59.376712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b535f3-2f4d-479f-80e7-22fd5a69c0cc None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-08b535f3-2f4d-479f-80e7-22fd5a69c0cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:37:59.390112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:59.392157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:59.394193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:59.394193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:59.394193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:37:59.394193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:37:59.394193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:37:59.394193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:37:59.394193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:37:59.394193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:37:59.394193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:37:59.394193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:37:59.394193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 343/694] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Aug 3 18:37:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:59.396905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b535f3-2f4d-479f-80e7-22fd5a69c0cc None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:59.404877 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143739f5-f3f9-495e-adcf-14abf74a3cbf None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:59.470681 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa9116-ecb0-41ce-a572-eb7af96fb281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:59.471905 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa9116-ecb0-41ce-a572-eb7af96fb281 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:59.473249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa9116-ecb0-41ce-a572-eb7af96fb281 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:59.473677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa9116-ecb0-41ce-a572-eb7af96fb281 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:59.483699 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa9116-ecb0-41ce-a572-eb7af96fb281 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:59.486200 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa9116-ecb0-41ce-a572-eb7af96fb281 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:59.504560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-85aa9116-ecb0-41ce-a572-eb7af96fb281 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:59.504560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:37:59.504560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:37:59.504560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:37:59.504560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:37:59.504560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:37:59.504560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:37:59.504560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:37:59.504560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:37:59.504560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:37:59.504560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:37:59.504560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:37:59.504560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:37:59.504560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:37:59.504560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:37:59.505505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:37:59.506393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:37:59.506393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:37:59.506393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:37:59.506393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:37:59.506393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:37:59.506393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:37:59.506393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:37:59.506393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:37:59.506901 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b535f3-2f4d-479f-80e7-22fd5a69c0cc None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:59.514346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 352/695] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:37:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:37:59.521412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa9116-ecb0-41ce-a572-eb7af96fb281 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:59.525476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:59.532267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:59.557300 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a4d0b-5e37-4520-bb5d-2013b93223b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:59.558378 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a4d0b-5e37-4520-bb5d-2013b93223b7 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:59.558899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a4d0b-5e37-4520-bb5d-2013b93223b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:59.559435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a4d0b-5e37-4520-bb5d-2013b93223b7 None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:59.560474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a4d0b-5e37-4520-bb5d-2013b93223b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:59.560999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a4d0b-5e37-4520-bb5d-2013b93223b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:59.562880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a4d0b-5e37-4520-bb5d-2013b93223b7 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:59.584976 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 344/696] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Aug 3 18:37:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:59.593103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a4d0b-5e37-4520-bb5d-2013b93223b7 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:59.763222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa9116-ecb0-41ce-a572-eb7af96fb281 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:59.791666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a4d0b-5e37-4520-bb5d-2013b93223b7 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:37:59.827157 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bca34-6049-4b69-95d2-8953006b829e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:59.827800 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bca34-6049-4b69-95d2-8953006b829e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:59.828080 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bca34-6049-4b69-95d2-8953006b829e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:59.828381 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bca34-6049-4b69-95d2-8953006b829e None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:59.829330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bca34-6049-4b69-95d2-8953006b829e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:37:59.829641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bca34-6049-4b69-95d2-8953006b829e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:37:59.831078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bca34-6049-4b69-95d2-8953006b829e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:37:59.850676 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 353/697] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:37:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:37:59.856105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0687d77-6585-455f-9a14-7f2b8bdda002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:37:59.856767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0687d77-6585-455f-9a14-7f2b8bdda002 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:37:59.857047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0687d77-6585-455f-9a14-7f2b8bdda002 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:37:59.857385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0687d77-6585-455f-9a14-7f2b8bdda002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:37:59.859058 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bca34-6049-4b69-95d2-8953006b829e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:37:59.948618 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:59.978458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:37:59.999202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0687d77-6585-455f-9a14-7f2b8bdda002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:00.000690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0687d77-6585-455f-9a14-7f2b8bdda002 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:00.050737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bca34-6049-4b69-95d2-8953006b829e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:00.130882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695182b-df10-4108-b392-b07fa0269e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:00.131512 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695182b-df10-4108-b392-b07fa0269e5a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:00.131914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695182b-df10-4108-b392-b07fa0269e5a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:00.132032 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695182b-df10-4108-b392-b07fa0269e5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:00.133770 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695182b-df10-4108-b392-b07fa0269e5a None admin] RBAC: Adding query filter params (role.id=d67cdb8bae3648fd8f3589ab33176d14, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=96ed059287274ca392c6abb893551094) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:00.133881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695182b-df10-4108-b392-b07fa0269e5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:00.135296 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695182b-df10-4108-b392-b07fa0269e5a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:00.152403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 354/698] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:37:59 2022] GET /identity/v3/role_assignments?user.id=96ed059287274ca392c6abb893551094&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=d67cdb8bae3648fd8f3589ab33176d14 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:00.159022 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 345/699] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:37:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:38:00.168456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695182b-df10-4108-b392-b07fa0269e5a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:00.193129 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba98164-ba08-4391-9579-60984f577d93 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:00.193459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba98164-ba08-4391-9579-60984f577d93 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:00.193692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba98164-ba08-4391-9579-60984f577d93 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:00.196240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 346/700] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:00.201847 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a92de-1378-4f76-b899-b676e1ca9bd5 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:00.202167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a92de-1378-4f76-b899-b676e1ca9bd5 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:00.202411 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a92de-1378-4f76-b899-b676e1ca9bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:00.222898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-357a92de-1378-4f76-b899-b676e1ca9bd5 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:00.224138 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-357a92de-1378-4f76-b899-b676e1ca9bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:00.319300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695182b-df10-4108-b392-b07fa0269e5a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:00.352561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:00.361962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc3a93-211f-432f-af31-9f97a5a28625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:00.362672 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc3a93-211f-432f-af31-9f97a5a28625 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:00.363023 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc3a93-211f-432f-af31-9f97a5a28625 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:00.363355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc3a93-211f-432f-af31-9f97a5a28625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:00.378428 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc3a93-211f-432f-af31-9f97a5a28625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:00.380064 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 347/701] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:38:00.384783 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc3a93-211f-432f-af31-9f97a5a28625 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:00.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-22bc3a93-211f-432f-af31-9f97a5a28625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:00.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:00.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:00.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:00.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:00.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:00.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:00.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:00.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:00.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:00.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:00.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:00.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:00.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:00.401720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:00.402708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:00.403734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:00.403734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:00.403734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:00.403734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:00.403734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 355/702] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:00.408537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a92de-1378-4f76-b899-b676e1ca9bd5 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:00.411737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:00.419606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc3a93-211f-432f-af31-9f97a5a28625 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:00.543030 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a92de-1378-4f76-b899-b676e1ca9bd5 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:00.583316 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc3a93-211f-432f-af31-9f97a5a28625 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:00.587956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e18ec-d16e-4fbb-b784-bb3c3958463b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:00.589073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e18ec-d16e-4fbb-b784-bb3c3958463b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:00.589547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e18ec-d16e-4fbb-b784-bb3c3958463b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:00.589996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e18ec-d16e-4fbb-b784-bb3c3958463b None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:00.590943 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e18ec-d16e-4fbb-b784-bb3c3958463b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:00.591430 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e18ec-d16e-4fbb-b784-bb3c3958463b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:00.593136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e18ec-d16e-4fbb-b784-bb3c3958463b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:00.609113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 348/703] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:00.614778 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e18ec-d16e-4fbb-b784-bb3c3958463b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:00.633726 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfab999-3a74-4177-aafc-5fc18b4f5786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:00.634658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfab999-3a74-4177-aafc-5fc18b4f5786 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:00.634946 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfab999-3a74-4177-aafc-5fc18b4f5786 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:00.635228 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfab999-3a74-4177-aafc-5fc18b4f5786 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:00.650692 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfab999-3a74-4177-aafc-5fc18b4f5786 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:00.653352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfab999-3a74-4177-aafc-5fc18b4f5786 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:00.674777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-1dfab999-3a74-4177-aafc-5fc18b4f5786 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:00.674777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:00.674777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:00.674777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:00.674777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:00.674777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:00.674777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:00.674777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:00.674777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:00.674777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:00.674777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:00.674777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:00.674777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:00.674777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:00.674777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:00.675954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:00.677309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:00.677309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:00.677309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:00.677309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:00.677309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 356/704] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:00.683140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfab999-3a74-4177-aafc-5fc18b4f5786 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:00.704211 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:00.719506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:00.775122 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e18ec-d16e-4fbb-b784-bb3c3958463b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:00.825140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfab999-3a74-4177-aafc-5fc18b4f5786 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:00.858314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6cd2c-e767-475b-a24e-d686d6d8e795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:00.858958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6cd2c-e767-475b-a24e-d686d6d8e795 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:00.859352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6cd2c-e767-475b-a24e-d686d6d8e795 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:00.859637 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6cd2c-e767-475b-a24e-d686d6d8e795 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:00.865266 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204c026-afb9-4ba2-9845-ce4b21cf7404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:00.865907 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204c026-afb9-4ba2-9845-ce4b21cf7404 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:00.866190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204c026-afb9-4ba2-9845-ce4b21cf7404 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:00.866468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204c026-afb9-4ba2-9845-ce4b21cf7404 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:00.867640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204c026-afb9-4ba2-9845-ce4b21cf7404 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:00.867944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204c026-afb9-4ba2-9845-ce4b21cf7404 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:00.869534 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204c026-afb9-4ba2-9845-ce4b21cf7404 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:00.877641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6cd2c-e767-475b-a24e-d686d6d8e795 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:00.879523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6cd2c-e767-475b-a24e-d686d6d8e795 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:00.883310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 357/705] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:00.888947 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204c026-afb9-4ba2-9845-ce4b21cf7404 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:00.905608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-8ac6cd2c-e767-475b-a24e-d686d6d8e795 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:00.905608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:00.905608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:00.905608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:00.905608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:00.905608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:00.905608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:00.905608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:00.905608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:00.905608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:00.905608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:00.905608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:00.905608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:00.905608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:00.905608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:00.906569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:00.907464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:00.907464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:00.907464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:00.907464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:00.907464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:00.907464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:00.907464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:00.907464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:00.914339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 349/706] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:00.922118 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6cd2c-e767-475b-a24e-d686d6d8e795 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:00.995145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:01.006134 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204c026-afb9-4ba2-9845-ce4b21cf7404 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:01.055970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c494313-dfb4-43a6-88c5-709acb95b967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:01.056737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c494313-dfb4-43a6-88c5-709acb95b967 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:01.057022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c494313-dfb4-43a6-88c5-709acb95b967 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:01.057295 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c494313-dfb4-43a6-88c5-709acb95b967 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:01.057988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c494313-dfb4-43a6-88c5-709acb95b967 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:01.059758 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c494313-dfb4-43a6-88c5-709acb95b967 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:01.084062 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 358/707] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:38:00 2022] POST /identity/v3/users => generated 254 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:38:01.086259 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab53f65-616d-4aab-af0c-2607e9aea93d None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:01.086687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab53f65-616d-4aab-af0c-2607e9aea93d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:01.086903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab53f65-616d-4aab-af0c-2607e9aea93d None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:01.092885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 359/708] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:01.094315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf75b8-1542-4806-920e-abde3b635ad5 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:01.094608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf75b8-1542-4806-920e-abde3b635ad5 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:01.094998 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf75b8-1542-4806-920e-abde3b635ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:01.102750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6cd2c-e767-475b-a24e-d686d6d8e795 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:01.118018 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-10cf75b8-1542-4806-920e-abde3b635ad5 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:01.119421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-10cf75b8-1542-4806-920e-abde3b635ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:01.128544 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:01.155944 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926e348-91d7-4c18-9c9a-4b2d20ac2b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:01.156811 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926e348-91d7-4c18-9c9a-4b2d20ac2b70 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:01.156990 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926e348-91d7-4c18-9c9a-4b2d20ac2b70 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:01.157167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926e348-91d7-4c18-9c9a-4b2d20ac2b70 None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:01.157957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926e348-91d7-4c18-9c9a-4b2d20ac2b70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:01.158267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926e348-91d7-4c18-9c9a-4b2d20ac2b70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:01.159761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926e348-91d7-4c18-9c9a-4b2d20ac2b70 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:01.188461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 350/709] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:38:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:01.194773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926e348-91d7-4c18-9c9a-4b2d20ac2b70 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:01.225214 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 360/710] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:38:01.257954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf75b8-1542-4806-920e-abde3b635ad5 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:01.352520 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:01.381118 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf75b8-1542-4806-920e-abde3b635ad5 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:01.391321 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926e348-91d7-4c18-9c9a-4b2d20ac2b70 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:01.427010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364ea46-05b1-40c3-9039-7c9fdff39a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:01.427756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364ea46-05b1-40c3-9039-7c9fdff39a98 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:01.428073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364ea46-05b1-40c3-9039-7c9fdff39a98 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:01.428429 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364ea46-05b1-40c3-9039-7c9fdff39a98 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:01.429697 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364ea46-05b1-40c3-9039-7c9fdff39a98 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:01.431583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364ea46-05b1-40c3-9039-7c9fdff39a98 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:01.440821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058721b-5a2b-49f4-b0f3-cc7332d8c3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:01.441775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058721b-5a2b-49f4-b0f3-cc7332d8c3f4 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:01.442268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058721b-5a2b-49f4-b0f3-cc7332d8c3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:01.442740 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058721b-5a2b-49f4-b0f3-cc7332d8c3f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:01.443738 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058721b-5a2b-49f4-b0f3-cc7332d8c3f4 None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=850a50cfe4394febbafb0908b08888b2) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:01.444264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058721b-5a2b-49f4-b0f3-cc7332d8c3f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:01.446744 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058721b-5a2b-49f4-b0f3-cc7332d8c3f4 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-e364ea46-05b1-40c3-9039-7c9fdff39a98 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:01.448756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:01.449751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:01.450651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 361/711] 10.222.0.69 () {56 vars in 1129 bytes} [Wed Aug 3 18:38:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:01.456678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364ea46-05b1-40c3-9039-7c9fdff39a98 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:01.458125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 351/712] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:38:01 2022] GET /identity/v3/role_assignments?user.id=850a50cfe4394febbafb0908b08888b2&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=19557e941d6f475ba92cf862846f9872 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:01.546226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968df5e-3514-40d8-8faa-dc288b00c693 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:01.546867 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968df5e-3514-40d8-8faa-dc288b00c693 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:01.547346 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968df5e-3514-40d8-8faa-dc288b00c693 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:01.549192 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 352/713] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:38:01.555097 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c70224-9336-4383-82b9-e1ae47c26d58 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:01.555850 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c70224-9336-4383-82b9-e1ae47c26d58 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:01.556352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c70224-9336-4383-82b9-e1ae47c26d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:01.592683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f6c70224-9336-4383-82b9-e1ae47c26d58 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:01.593379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364ea46-05b1-40c3-9039-7c9fdff39a98 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:01.594462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f6c70224-9336-4383-82b9-e1ae47c26d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:01.636603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69f9cd-8362-4b3e-9af2-e5a97faa5162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:01.637215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69f9cd-8362-4b3e-9af2-e5a97faa5162 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:01.637554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69f9cd-8362-4b3e-9af2-e5a97faa5162 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:01.637905 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69f9cd-8362-4b3e-9af2-e5a97faa5162 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:01.640319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69f9cd-8362-4b3e-9af2-e5a97faa5162 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:01.640319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69f9cd-8362-4b3e-9af2-e5a97faa5162 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:01.641119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69f9cd-8362-4b3e-9af2-e5a97faa5162 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:01.647725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 362/714] 10.222.0.69 () {56 vars in 1124 bytes} [Wed Aug 3 18:38:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:01.654806 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69f9cd-8362-4b3e-9af2-e5a97faa5162 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:01.659142 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:01.699918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:01.737332 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 353/715] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:38:01.767023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c70224-9336-4383-82b9-e1ae47c26d58 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:01.804992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69f9cd-8362-4b3e-9af2-e5a97faa5162 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:01.857340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460586c4-2334-4709-b254-a188e62651bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:01.857976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460586c4-2334-4709-b254-a188e62651bc None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:01.858261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460586c4-2334-4709-b254-a188e62651bc None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:01.858536 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460586c4-2334-4709-b254-a188e62651bc None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:01.859233 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460586c4-2334-4709-b254-a188e62651bc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:01.859535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460586c4-2334-4709-b254-a188e62651bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:01.860952 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460586c4-2334-4709-b254-a188e62651bc None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:01.868330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 363/716] 10.222.0.69 () {56 vars in 1124 bytes} [Wed Aug 3 18:38:01 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:01.956511 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c70224-9336-4383-82b9-e1ae47c26d58 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:02.015651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33525e87-2f5f-4322-ada5-1aef01d0a552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:02.016344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33525e87-2f5f-4322-ada5-1aef01d0a552 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:02.016632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33525e87-2f5f-4322-ada5-1aef01d0a552 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:02.016900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33525e87-2f5f-4322-ada5-1aef01d0a552 None admin] PATH_INFO: `/v3/services/network` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:02.017541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33525e87-2f5f-4322-ada5-1aef01d0a552 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:02.018980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33525e87-2f5f-4322-ada5-1aef01d0a552 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-33525e87-2f5f-4322-ada5-1aef01d0a552 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:02.033727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:02.034817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:02.035684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:02.035684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:02.035684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:02.035684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:02.035684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:02.035684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:02.035684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:02.035684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:02.035684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:02.035684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:02.035684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:02.035684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:02.035684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 18:38:02.035684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:02.035684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 354/717] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:01 2022] GET /identity/v3/services/network => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:02.042660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460586c4-2334-4709-b254-a188e62651bc None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:02.123991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:02.162610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460586c4-2334-4709-b254-a188e62651bc None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:02.206436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a044d3-8b30-40a3-93fb-12556a835482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:02.207451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a044d3-8b30-40a3-93fb-12556a835482 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:02.207959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a044d3-8b30-40a3-93fb-12556a835482 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:02.208518 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a044d3-8b30-40a3-93fb-12556a835482 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:02.209451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a044d3-8b30-40a3-93fb-12556a835482 None admin] RBAC: Adding query filter params (name=network) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:02.209966 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a044d3-8b30-40a3-93fb-12556a835482 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:02.211753 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a044d3-8b30-40a3-93fb-12556a835482 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:02.220018 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 364/718] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Aug 3 18:38:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:02.225616 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33525e87-2f5f-4322-ada5-1aef01d0a552 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:02.298187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:02.368993 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33525e87-2f5f-4322-ada5-1aef01d0a552 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:02.423844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaaabb-ab5c-44d3-a64e-160bca226f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:02.424511 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaaabb-ab5c-44d3-a64e-160bca226f1b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:02.424794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaaabb-ab5c-44d3-a64e-160bca226f1b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:02.425070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaaabb-ab5c-44d3-a64e-160bca226f1b None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:02.425768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaaabb-ab5c-44d3-a64e-160bca226f1b None admin] RBAC: Adding query filter params (type=network) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:02.426071 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaaabb-ab5c-44d3-a64e-160bca226f1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:02.427448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaaabb-ab5c-44d3-a64e-160bca226f1b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:02.435632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 355/719] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Aug 3 18:38:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:03.187698 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9c793-220a-490e-9ab1-64b99c979c3f None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:03.188194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9c793-220a-490e-9ab1-64b99c979c3f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:03.188194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9c793-220a-490e-9ab1-64b99c979c3f None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:03.189875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 365/720] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:03.195965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca10ed0-4159-46af-982f-cb304314230c None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:03.196344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca10ed0-4159-46af-982f-cb304314230c None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:03.196759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca10ed0-4159-46af-982f-cb304314230c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:03.215694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-1ca10ed0-4159-46af-982f-cb304314230c None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:03.216927 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1ca10ed0-4159-46af-982f-cb304314230c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:03.326526 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 356/721] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 18:38:03.366081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9c793-220a-490e-9ab1-64b99c979c3f None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:03.512495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9c793-220a-490e-9ab1-64b99c979c3f None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:03.554890 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:03.560802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c8244-c844-4880-ae3b-c90a8de0867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:03.561495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c8244-c844-4880-ae3b-c90a8de0867e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:03.561809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c8244-c844-4880-ae3b-c90a8de0867e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:03.562092 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c8244-c844-4880-ae3b-c90a8de0867e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:03.575801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c8244-c844-4880-ae3b-c90a8de0867e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-822c8244-c844-4880-ae3b-c90a8de0867e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:03.580895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:03.581834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:03.582683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:03.582683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:03.582683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:03.582683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:03.582683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:03.582683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:03.582683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:03.582683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:03.582683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:03.582683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:03.582683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:03.582683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:03.582683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 366/722] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:38:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:03.589650 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca10ed0-4159-46af-982f-cb304314230c None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:03.635649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:03.716483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca10ed0-4159-46af-982f-cb304314230c None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:03.755005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d901c8-f94b-4618-be23-02aaeb091c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:03.755632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d901c8-f94b-4618-be23-02aaeb091c0f None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:03.755918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d901c8-f94b-4618-be23-02aaeb091c0f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:03.756197 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d901c8-f94b-4618-be23-02aaeb091c0f None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:03.756955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d901c8-f94b-4618-be23-02aaeb091c0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:03.757375 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d901c8-f94b-4618-be23-02aaeb091c0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:03.758926 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d901c8-f94b-4618-be23-02aaeb091c0f None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:03.771565 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 357/723] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:38:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:03.778004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c8244-c844-4880-ae3b-c90a8de0867e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:03.906665 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17910ee4-26c4-4b2d-a897-37efdc6c0fce None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:03.907021 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17910ee4-26c4-4b2d-a897-37efdc6c0fce None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:03.907284 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17910ee4-26c4-4b2d-a897-37efdc6c0fce None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:03.908905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 358/724] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:03.917066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c8244-c844-4880-ae3b-c90a8de0867e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:03.918908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc721e-a3d0-4afd-84d0-688f8e6c17c4 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:03.919203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc721e-a3d0-4afd-84d0-688f8e6c17c4 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:03.919485 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc721e-a3d0-4afd-84d0-688f8e6c17c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:03.944213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-89bc721e-a3d0-4afd-84d0-688f8e6c17c4 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:03.946488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-89bc721e-a3d0-4afd-84d0-688f8e6c17c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:03.973704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f896b0-6276-4565-bad6-882665ce6811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:03.974516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f896b0-6276-4565-bad6-882665ce6811 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:03.974875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f896b0-6276-4565-bad6-882665ce6811 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:03.975351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f896b0-6276-4565-bad6-882665ce6811 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:03.989759 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f896b0-6276-4565-bad6-882665ce6811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:03.991311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f896b0-6276-4565-bad6-882665ce6811 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:04.012114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-30f896b0-6276-4565-bad6-882665ce6811 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:04.012114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:04.012114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:04.012114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:04.012114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:04.012114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:04.012114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:04.012114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:04.012114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:04.012114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:04.012114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:04.012114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:04.012114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:04.012114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:04.012114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:04.013120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:04.014039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:04.014039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:04.014039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:04.014039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:04.014039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 367/725] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:04.016849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362b896-8c5a-467a-b7c4-04924b6fe98b None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:04.017325 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362b896-8c5a-467a-b7c4-04924b6fe98b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:04.017773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362b896-8c5a-467a-b7c4-04924b6fe98b None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:04.019332 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 368/726] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:04.021295 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362b896-8c5a-467a-b7c4-04924b6fe98b None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:04.052473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 359/727] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:38:04.054964 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e5e63-7a51-442f-a087-c690f56c6030 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:04.054964 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e5e63-7a51-442f-a087-c690f56c6030 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:04.055315 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e5e63-7a51-442f-a087-c690f56c6030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:04.082747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:04.087716 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-017e5e63-7a51-442f-a087-c690f56c6030 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:04.089081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-017e5e63-7a51-442f-a087-c690f56c6030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:04.111650 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:04.149912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362b896-8c5a-467a-b7c4-04924b6fe98b None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:04.172651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 360/728] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:38:04.173640 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e5e63-7a51-442f-a087-c690f56c6030 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:04.188959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854c7ed-b804-418d-a604-4e4c1819a3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:04.190297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854c7ed-b804-418d-a604-4e4c1819a3a4 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:04.190835 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854c7ed-b804-418d-a604-4e4c1819a3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:04.191314 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854c7ed-b804-418d-a604-4e4c1819a3a4 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:04.192267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854c7ed-b804-418d-a604-4e4c1819a3a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:04.192819 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854c7ed-b804-418d-a604-4e4c1819a3a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:04.194552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854c7ed-b804-418d-a604-4e4c1819a3a4 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:04.212865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 369/729] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:04.214128 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854c7ed-b804-418d-a604-4e4c1819a3a4 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:04.294742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e5e63-7a51-442f-a087-c690f56c6030 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:04.330007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854c7ed-b804-418d-a604-4e4c1819a3a4 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:04.331392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b8f0d-e5d8-42a5-9380-5f9c879f3d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:04.332047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b8f0d-e5d8-42a5-9380-5f9c879f3d3a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:04.332356 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b8f0d-e5d8-42a5-9380-5f9c879f3d3a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:04.332638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b8f0d-e5d8-42a5-9380-5f9c879f3d3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:04.334364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:04.345951 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b8f0d-e5d8-42a5-9380-5f9c879f3d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-cf4b8f0d-e5d8-42a5-9380-5f9c879f3d3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:04.350988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:04.352531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:04.353803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:04.353803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:04.353803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:04.353803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:04.353803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:04.353803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:04.353803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:04.353803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:04.353803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:04.353803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:04.353803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:38:04.353803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:04.353803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 361/730] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:38:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:04.357296 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b8f0d-e5d8-42a5-9380-5f9c879f3d3a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:04.377702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:04.386806 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d22115-7feb-4c5b-91ca-37f07dd73b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:04.387783 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d22115-7feb-4c5b-91ca-37f07dd73b01 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:04.388272 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d22115-7feb-4c5b-91ca-37f07dd73b01 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:04.388846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d22115-7feb-4c5b-91ca-37f07dd73b01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:04.401908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d22115-7feb-4c5b-91ca-37f07dd73b01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-f5d22115-7feb-4c5b-91ca-37f07dd73b01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:04.408099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:04.409057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:04.409936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:04.409936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:04.409936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:04.409936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:04.409936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:04.409936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:04.409936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:04.409936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:04.409936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:04.409936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:04.409936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:04.409936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:04.412786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 370/731] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:38:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:04.414290 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d22115-7feb-4c5b-91ca-37f07dd73b01 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:04.471667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b8f0d-e5d8-42a5-9380-5f9c879f3d3a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:04.511151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a162e34-f101-4891-b07c-4ca38f9b6c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:04.511801 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a162e34-f101-4891-b07c-4ca38f9b6c70 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:04.512135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a162e34-f101-4891-b07c-4ca38f9b6c70 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:04.512448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a162e34-f101-4891-b07c-4ca38f9b6c70 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:04.520029 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a162e34-f101-4891-b07c-4ca38f9b6c70 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:04.521525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a162e34-f101-4891-b07c-4ca38f9b6c70 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:04.530263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d22115-7feb-4c5b-91ca-37f07dd73b01 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-3a162e34-f101-4891-b07c-4ca38f9b6c70 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:38:04.535343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:04.536274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:04.537484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:04.537484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:04.537484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:38:04.537484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:38:04.537484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:38:04.537484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:38:04.537484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:38:04.537484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:38:04.537484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 18:38:04.537484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:04.537484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 362/732] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:38:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:04.538081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a162e34-f101-4891-b07c-4ca38f9b6c70 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:04.572484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecef8fc-46a4-4bba-a25e-9bd0a7f0e43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:04.573121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecef8fc-46a4-4bba-a25e-9bd0a7f0e43b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:04.573405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecef8fc-46a4-4bba-a25e-9bd0a7f0e43b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:04.573677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecef8fc-46a4-4bba-a25e-9bd0a7f0e43b None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:04.574409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecef8fc-46a4-4bba-a25e-9bd0a7f0e43b None admin] RBAC: Adding query filter params (name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:04.574719 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecef8fc-46a4-4bba-a25e-9bd0a7f0e43b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:04.576106 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecef8fc-46a4-4bba-a25e-9bd0a7f0e43b None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:04.585170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 371/733] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:38:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:04.586743 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2e25b-74ca-475a-9c73-1c78d8c767a7 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:04.587034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2e25b-74ca-475a-9c73-1c78d8c767a7 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:04.587314 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2e25b-74ca-475a-9c73-1c78d8c767a7 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:04.588764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 372/734] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:04.589308 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c2e25b-74ca-475a-9c73-1c78d8c767a7 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:04.621710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:04.635666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:04.663903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a162e34-f101-4891-b07c-4ca38f9b6c70 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:04.703085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0c1ee-f8bd-477f-8844-51714b52c13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:04.703709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0c1ee-f8bd-477f-8844-51714b52c13a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:04.703997 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0c1ee-f8bd-477f-8844-51714b52c13a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:04.704274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0c1ee-f8bd-477f-8844-51714b52c13a None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:04.705030 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0c1ee-f8bd-477f-8844-51714b52c13a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:04.705130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c2e25b-74ca-475a-9c73-1c78d8c767a7 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:04.705336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0c1ee-f8bd-477f-8844-51714b52c13a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:04.706728 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0c1ee-f8bd-477f-8844-51714b52c13a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:04.720810 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 363/735] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:38:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:04.721574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0c1ee-f8bd-477f-8844-51714b52c13a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:04.742275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90582de-b118-4db0-819e-3efe57e9ae4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:04.743039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90582de-b118-4db0-819e-3efe57e9ae4f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:04.743326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90582de-b118-4db0-819e-3efe57e9ae4f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:04.743606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90582de-b118-4db0-819e-3efe57e9ae4f None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:04.744455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90582de-b118-4db0-819e-3efe57e9ae4f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:04.744767 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90582de-b118-4db0-819e-3efe57e9ae4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:04.746226 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90582de-b118-4db0-819e-3efe57e9ae4f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:04.769514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 373/736] 10.222.0.69 () {56 vars in 1135 bytes} [Wed Aug 3 18:38:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:04.769514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54201d-6ab8-4f8d-827e-f770a1f5e60e None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:04.769514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54201d-6ab8-4f8d-827e-f770a1f5e60e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:04.769514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54201d-6ab8-4f8d-827e-f770a1f5e60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:04.788248 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-fb54201d-6ab8-4f8d-827e-f770a1f5e60e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:04.790266 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-fb54201d-6ab8-4f8d-827e-f770a1f5e60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:04.840596 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0c1ee-f8bd-477f-8844-51714b52c13a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:04.872314 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 374/737] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:38:04.872314 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54201d-6ab8-4f8d-827e-f770a1f5e60e None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:04.878704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:04.891654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c001d-3223-4f6a-a07c-34e77d515cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:04.892429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c001d-3223-4f6a-a07c-34e77d515cc2 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:04.892822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c001d-3223-4f6a-a07c-34e77d515cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:04.893108 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c001d-3223-4f6a-a07c-34e77d515cc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:04.901446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:04.906095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c001d-3223-4f6a-a07c-34e77d515cc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:04.907958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c001d-3223-4f6a-a07c-34e77d515cc2 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:04.925294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-bd6c001d-3223-4f6a-a07c-34e77d515cc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:04.925294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:04.925294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:04.925294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:04.925294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:04.925294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:04.925294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:04.925294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:04.925294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:04.925294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:04.925294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:04.925294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:04.925294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:04.925294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:04.925294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:04.926263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:04.927218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:04.927218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:04.927218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:04.927218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:04.927218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 364/738] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:04.927218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c001d-3223-4f6a-a07c-34e77d515cc2 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:04.982178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54201d-6ab8-4f8d-827e-f770a1f5e60e None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:05.018008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f717ab8-7e67-4353-81da-9da6dd806963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:05.018661 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f717ab8-7e67-4353-81da-9da6dd806963 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:05.018944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f717ab8-7e67-4353-81da-9da6dd806963 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:05.019222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f717ab8-7e67-4353-81da-9da6dd806963 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:05.029535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f717ab8-7e67-4353-81da-9da6dd806963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:05.031024 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f717ab8-7e67-4353-81da-9da6dd806963 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:05.045576 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c001d-3223-4f6a-a07c-34e77d515cc2 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:05.046275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-5f717ab8-7e67-4353-81da-9da6dd806963 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:05.046275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:05.046275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:05.046275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:05.046275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:05.046275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:05.046275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:05.046275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:05.046275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:05.046275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:05.046275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:05.046275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:05.046275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:05.046275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:05.046275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:05.047234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:05.048173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:05.048173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:05.048173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:05.048173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:05.048173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 375/739] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:05.048506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f717ab8-7e67-4353-81da-9da6dd806963 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:05.084182 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13317d-2228-44d8-b8a9-d475201171d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:05.084839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13317d-2228-44d8-b8a9-d475201171d2 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:05.085125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13317d-2228-44d8-b8a9-d475201171d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:05.085549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13317d-2228-44d8-b8a9-d475201171d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:05.117622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:05.146048 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:05.156798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f717ab8-7e67-4353-81da-9da6dd806963 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:05.157624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13317d-2228-44d8-b8a9-d475201171d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:05.159201 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13317d-2228-44d8-b8a9-d475201171d2 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:05.191261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713e9c0-312a-43af-aa3d-95e758c5120a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:05.191885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713e9c0-312a-43af-aa3d-95e758c5120a None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:05.192173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713e9c0-312a-43af-aa3d-95e758c5120a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:05.192484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713e9c0-312a-43af-aa3d-95e758c5120a None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:05.193182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713e9c0-312a-43af-aa3d-95e758c5120a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:05.194552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713e9c0-312a-43af-aa3d-95e758c5120a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:05.211236 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 376/740] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Aug 3 18:38:05 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:38:05.212057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713e9c0-312a-43af-aa3d-95e758c5120a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:05.292654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 365/741] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:38:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 18:38:05.294750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4567d-1a63-4614-81ea-7b2882dcfc9a None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:05.295051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4567d-1a63-4614-81ea-7b2882dcfc9a None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:05.295329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4567d-1a63-4614-81ea-7b2882dcfc9a None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:05.297087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 366/742] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:05.297710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4567d-1a63-4614-81ea-7b2882dcfc9a None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:05.355168 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713e9c0-312a-43af-aa3d-95e758c5120a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:05.382010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:05.400166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7caf7c-d539-4132-95aa-36a30b48c7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:05.401244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7caf7c-d539-4132-95aa-36a30b48c7f4 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:05.401530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7caf7c-d539-4132-95aa-36a30b48c7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:05.402200 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7caf7c-d539-4132-95aa-36a30b48c7f4 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:05.403218 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7caf7c-d539-4132-95aa-36a30b48c7f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:05.403530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7caf7c-d539-4132-95aa-36a30b48c7f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:05.405414 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7caf7c-d539-4132-95aa-36a30b48c7f4 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:05.410462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:05.419908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 377/743] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:05.420803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7caf7c-d539-4132-95aa-36a30b48c7f4 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:05.425185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4567d-1a63-4614-81ea-7b2882dcfc9a None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:05.462026 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d70bfc-62e5-464b-ab1f-a30afda29ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:05.462645 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d70bfc-62e5-464b-ab1f-a30afda29ed8 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:05.462924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d70bfc-62e5-464b-ab1f-a30afda29ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:05.463211 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d70bfc-62e5-464b-ab1f-a30afda29ed8 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:05.463927 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d70bfc-62e5-464b-ab1f-a30afda29ed8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:05.464230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d70bfc-62e5-464b-ab1f-a30afda29ed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:05.465620 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d70bfc-62e5-464b-ab1f-a30afda29ed8 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:05.477701 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 367/744] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:05.479089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134838-b4ed-4387-82f9-078628c22bed None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:05.479379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134838-b4ed-4387-82f9-078628c22bed None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:05.479652 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134838-b4ed-4387-82f9-078628c22bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:05.499180 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-aa134838-b4ed-4387-82f9-078628c22bed None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:05.500392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-aa134838-b4ed-4387-82f9-078628c22bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:05.537702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7caf7c-d539-4132-95aa-36a30b48c7f4 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:05.580313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 368/745] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 18:38:05.580514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13974d7-0f03-4c78-91bd-b683cd055ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:05.580624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa134838-b4ed-4387-82f9-078628c22bed None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:05.580729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13974d7-0f03-4c78-91bd-b683cd055ad8 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:05.580729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13974d7-0f03-4c78-91bd-b683cd055ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:05.580729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13974d7-0f03-4c78-91bd-b683cd055ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:05.596617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13974d7-0f03-4c78-91bd-b683cd055ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:05.598889 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13974d7-0f03-4c78-91bd-b683cd055ad8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:05.615910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-c13974d7-0f03-4c78-91bd-b683cd055ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:05.615910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:05.615910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:05.615910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:05.615910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:05.615910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:05.615910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:05.615910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:05.615910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:05.615910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:05.615910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:05.615910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:05.615910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:05.615910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:05.615910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:05.617590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:05.618538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:05.618538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:05.618538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:05.618538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:05.618538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 378/746] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:05.618538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13974d7-0f03-4c78-91bd-b683cd055ad8 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:05.655052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:05.673430 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:05.701535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa134838-b4ed-4387-82f9-078628c22bed None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:05.728460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13974d7-0f03-4c78-91bd-b683cd055ad8 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:05.739225 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b9b55-f99f-4cf5-b0a8-5b5f62307d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:05.739877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b9b55-f99f-4cf5-b0a8-5b5f62307d4d None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:05.740160 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b9b55-f99f-4cf5-b0a8-5b5f62307d4d None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:05.740494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b9b55-f99f-4cf5-b0a8-5b5f62307d4d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:05.748598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b9b55-f99f-4cf5-b0a8-5b5f62307d4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:05.750100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b9b55-f99f-4cf5-b0a8-5b5f62307d4d None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-675b9b55-f99f-4cf5-b0a8-5b5f62307d4d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:38:05.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:05.765099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:05.766041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:05.766041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:05.766041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:38:05.766041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:38:05.766041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:38:05.766041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:38:05.766041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:38:05.766041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:38:05.766041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 18:38:05.766041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:05.766041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 369/747] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Aug 3 18:38:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:05.767225 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b9b55-f99f-4cf5-b0a8-5b5f62307d4d None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:05.771335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa154dc8-e824-45d8-9e3b-008c4ab8a0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:05.771992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa154dc8-e824-45d8-9e3b-008c4ab8a0b2 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:05.772279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa154dc8-e824-45d8-9e3b-008c4ab8a0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:05.772599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa154dc8-e824-45d8-9e3b-008c4ab8a0b2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:05.781634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa154dc8-e824-45d8-9e3b-008c4ab8a0b2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:05.783089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa154dc8-e824-45d8-9e3b-008c4ab8a0b2 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-aa154dc8-e824-45d8-9e3b-008c4ab8a0b2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:38:05.796199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:05.797409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:05.798488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:05.798488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:05.798488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:38:05.798488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:38:05.798488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:38:05.798488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:38:05.798488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:38:05.798488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:38:05.798488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:38:05.798488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:05.798488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 379/748] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Aug 3 18:38:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:05.802558 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa154dc8-e824-45d8-9e3b-008c4ab8a0b2 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:05.881895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b9b55-f99f-4cf5-b0a8-5b5f62307d4d None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:05.904991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:05.912341 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa154dc8-e824-45d8-9e3b-008c4ab8a0b2 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:05.926292 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4977d-5099-452d-839f-4ddc18ee15ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:05.927174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4977d-5099-452d-839f-4ddc18ee15ee None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:05.927605 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4977d-5099-452d-839f-4ddc18ee15ee None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:05.928030 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4977d-5099-452d-839f-4ddc18ee15ee None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:05.928921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f4977d-5099-452d-839f-4ddc18ee15ee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:05.930982 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f4977d-5099-452d-839f-4ddc18ee15ee None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:05.943632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:05.951877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 370/749] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Aug 3 18:38:05 2022] POST /identity/v3/services => generated 240 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:38:05.952766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4977d-5099-452d-839f-4ddc18ee15ee None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:05.977204 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ba29a-ff42-4fa4-bbc9-2d0d33d7a27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:05.978352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ba29a-ff42-4fa4-bbc9-2d0d33d7a27e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:05.978833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ba29a-ff42-4fa4-bbc9-2d0d33d7a27e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:05.979310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ba29a-ff42-4fa4-bbc9-2d0d33d7a27e None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:05.980229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ba29a-ff42-4fa4-bbc9-2d0d33d7a27e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:05.980745 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ba29a-ff42-4fa4-bbc9-2d0d33d7a27e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:05.982551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ba29a-ff42-4fa4-bbc9-2d0d33d7a27e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:06.002075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 380/750] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:06.003567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ba29a-ff42-4fa4-bbc9-2d0d33d7a27e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:06.076430 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4977d-5099-452d-839f-4ddc18ee15ee None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:06.115188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28889ceb-506d-4b9c-9e45-0b6a3fc0b22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:06.115830 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28889ceb-506d-4b9c-9e45-0b6a3fc0b22d None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:06.116112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28889ceb-506d-4b9c-9e45-0b6a3fc0b22d None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:06.116439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28889ceb-506d-4b9c-9e45-0b6a3fc0b22d None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:06.117317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28889ceb-506d-4b9c-9e45-0b6a3fc0b22d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:06.117687 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28889ceb-506d-4b9c-9e45-0b6a3fc0b22d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:06.119170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28889ceb-506d-4b9c-9e45-0b6a3fc0b22d None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:06.138602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 371/751] 10.222.0.69 () {56 vars in 1149 bytes} [Wed Aug 3 18:38:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:06.139492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28889ceb-506d-4b9c-9e45-0b6a3fc0b22d None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:06.155552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ba29a-ff42-4fa4-bbc9-2d0d33d7a27e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:06.200123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdad46b-e5e0-43e0-8f63-83cfc4d186f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:06.200773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdad46b-e5e0-43e0-8f63-83cfc4d186f0 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:06.200993 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdad46b-e5e0-43e0-8f63-83cfc4d186f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:06.201343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdad46b-e5e0-43e0-8f63-83cfc4d186f0 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:06.202368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdad46b-e5e0-43e0-8f63-83cfc4d186f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:06.202368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdad46b-e5e0-43e0-8f63-83cfc4d186f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:06.203777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdad46b-e5e0-43e0-8f63-83cfc4d186f0 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:06.218478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:06.221914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 381/752] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Aug 3 18:38:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:06.222762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdad46b-e5e0-43e0-8f63-83cfc4d186f0 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:06.238664 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:06.267788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28889ceb-506d-4b9c-9e45-0b6a3fc0b22d None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:06.303632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04c44d-02cf-4af9-a51a-c06a2bae4e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:06.304913 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04c44d-02cf-4af9-a51a-c06a2bae4e93 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:06.305200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04c44d-02cf-4af9-a51a-c06a2bae4e93 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:06.305588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04c44d-02cf-4af9-a51a-c06a2bae4e93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:06.312690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04c44d-02cf-4af9-a51a-c06a2bae4e93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:06.314143 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04c44d-02cf-4af9-a51a-c06a2bae4e93 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:06.326522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-5b04c44d-02cf-4af9-a51a-c06a2bae4e93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:06.326522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:06.326522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:06.326522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:06.326522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:06.326522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:06.326522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:06.326522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:06.326522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:06.326522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:06.326522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:06.326522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:06.326522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:06.326522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:06.326522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:06.327460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:06.328413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:06.328413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:06.328413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:06.328413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:06.328413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:06.328413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:06.328413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:06.328413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:06.328413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 372/753] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:06.328912 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04c44d-02cf-4af9-a51a-c06a2bae4e93 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:06.364711 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdad46b-e5e0-43e0-8f63-83cfc4d186f0 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:06.417963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad2957-accd-4fd3-8da4-4ed38c9ea85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:06.418974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad2957-accd-4fd3-8da4-4ed38c9ea85d None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:06.419459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad2957-accd-4fd3-8da4-4ed38c9ea85d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:06.419933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad2957-accd-4fd3-8da4-4ed38c9ea85d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:06.449034 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04c44d-02cf-4af9-a51a-c06a2bae4e93 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:06.467768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:06.487730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb224f1-d838-46c3-ad30-13f8bf3e65ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:06.488564 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb224f1-d838-46c3-ad30-13f8bf3e65ca None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:06.489002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb224f1-d838-46c3-ad30-13f8bf3e65ca None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:06.489376 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb224f1-d838-46c3-ad30-13f8bf3e65ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:06.502494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb224f1-d838-46c3-ad30-13f8bf3e65ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:06.504448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb224f1-d838-46c3-ad30-13f8bf3e65ca None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:06.515112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad2957-accd-4fd3-8da4-4ed38c9ea85d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:06.517585 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad2957-accd-4fd3-8da4-4ed38c9ea85d None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:06.521770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-4bb224f1-d838-46c3-ad30-13f8bf3e65ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:06.521770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:06.521770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:06.521770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:06.521770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:06.521770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:06.521770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:06.521770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:06.521770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:06.521770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:06.521770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:06.521770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:06.521770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:06.521770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:06.521770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:06.522642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:06.524089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:06.524089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:06.524089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:06.524089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:06.528656 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 373/754] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:06.529531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb224f1-d838-46c3-ad30-13f8bf3e65ca None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:06.534030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:06.634523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb224f1-d838-46c3-ad30-13f8bf3e65ca None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:06.657761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 382/755] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:38:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:38:06.659084 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad2957-accd-4fd3-8da4-4ed38c9ea85d None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:06.676656 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dc681-e842-4b26-914f-259d4ca44a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:06.677289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dc681-e842-4b26-914f-259d4ca44a42 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:06.677576 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dc681-e842-4b26-914f-259d4ca44a42 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:06.677859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dc681-e842-4b26-914f-259d4ca44a42 None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:06.678634 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1dc681-e842-4b26-914f-259d4ca44a42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:06.679822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1dc681-e842-4b26-914f-259d4ca44a42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:06.681701 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1dc681-e842-4b26-914f-259d4ca44a42 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:06.696424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 374/756] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:38:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:06.697720 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dc681-e842-4b26-914f-259d4ca44a42 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:06.733367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:06.803283 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad2957-accd-4fd3-8da4-4ed38c9ea85d None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:06.813117 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dc681-e842-4b26-914f-259d4ca44a42 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:06.814653 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:06.844798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a90d05-e733-4360-8073-9b8b7b260a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:06.845955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a90d05-e733-4360-8073-9b8b7b260a53 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:06.846459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a90d05-e733-4360-8073-9b8b7b260a53 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:06.846927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a90d05-e733-4360-8073-9b8b7b260a53 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:06.847857 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a90d05-e733-4360-8073-9b8b7b260a53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:06.848380 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a90d05-e733-4360-8073-9b8b7b260a53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:06.849958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a90d05-e733-4360-8073-9b8b7b260a53 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:06.850083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583cffa-774e-4d49-b945-ee2b55f31110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:06.850678 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583cffa-774e-4d49-b945-ee2b55f31110 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:06.850996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583cffa-774e-4d49-b945-ee2b55f31110 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:06.851419 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583cffa-774e-4d49-b945-ee2b55f31110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:06.862138 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583cffa-774e-4d49-b945-ee2b55f31110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:06.863591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583cffa-774e-4d49-b945-ee2b55f31110 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:06.864639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 383/757] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:06.865842 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a90d05-e733-4360-8073-9b8b7b260a53 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:06.885611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-3583cffa-774e-4d49-b945-ee2b55f31110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:06.885611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:06.885611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:06.885611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:06.885611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:06.885611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:06.885611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:06.885611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:06.885611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:06.885611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:06.885611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:06.885611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:06.885611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:06.885611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:06.885611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:06.886577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:06.887998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:06.887998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:06.887998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:06.887998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:06.891119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 375/758] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:06.891965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583cffa-774e-4d49-b945-ee2b55f31110 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:06.982728 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:06.998320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a90d05-e733-4360-8073-9b8b7b260a53 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:07.002793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583cffa-774e-4d49-b945-ee2b55f31110 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:07.037759 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aef69-1b86-4e8b-9c1d-159d0441df0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:07.038812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aef69-1b86-4e8b-9c1d-159d0441df0f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:07.039331 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aef69-1b86-4e8b-9c1d-159d0441df0f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:07.039476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db506a-7af7-4012-ad6a-7c1e91a50f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:07.039978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aef69-1b86-4e8b-9c1d-159d0441df0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:07.040084 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db506a-7af7-4012-ad6a-7c1e91a50f42 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:07.040306 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db506a-7af7-4012-ad6a-7c1e91a50f42 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:07.040599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db506a-7af7-4012-ad6a-7c1e91a50f42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:07.041547 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38aef69-1b86-4e8b-9c1d-159d0441df0f None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=96ed059287274ca392c6abb893551094) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:07.042057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38aef69-1b86-4e8b-9c1d-159d0441df0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:07.043922 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38aef69-1b86-4e8b-9c1d-159d0441df0f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:07.047890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5db506a-7af7-4012-ad6a-7c1e91a50f42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:07.049353 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5db506a-7af7-4012-ad6a-7c1e91a50f42 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:07.055793 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 384/759] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:38:06 2022] GET /identity/v3/role_assignments?user.id=96ed059287274ca392c6abb893551094&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=19557e941d6f475ba92cf862846f9872 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:07.057142 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aef69-1b86-4e8b-9c1d-159d0441df0f None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:07.061987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-a5db506a-7af7-4012-ad6a-7c1e91a50f42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:07.061987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:07.061987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:07.061987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:07.061987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:07.061987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:07.061987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:07.061987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:07.061987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:07.061987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:07.061987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:07.061987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:07.061987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:07.061987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:07.061987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:07.062921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:07.064237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:07.064237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:07.064237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:07.064237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:07.064237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:07.064237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:07.064237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:07.064237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:07.070909 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 376/760] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:07.070909 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db506a-7af7-4012-ad6a-7c1e91a50f42 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:07.115823 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:07.199446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db506a-7af7-4012-ad6a-7c1e91a50f42 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:07.253852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aef69-1b86-4e8b-9c1d-159d0441df0f None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:07.279784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76939f21-48bc-45f2-967d-1298e2d223f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:07.280586 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76939f21-48bc-45f2-967d-1298e2d223f5 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:07.280879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76939f21-48bc-45f2-967d-1298e2d223f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:07.281154 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76939f21-48bc-45f2-967d-1298e2d223f5 None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:07.281933 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76939f21-48bc-45f2-967d-1298e2d223f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:07.282242 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76939f21-48bc-45f2-967d-1298e2d223f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:07.283699 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76939f21-48bc-45f2-967d-1298e2d223f5 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:07.299915 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 377/761] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:38:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:07.306786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76939f21-48bc-45f2-967d-1298e2d223f5 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:07.316458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:07.316628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1f6d5-99cf-4815-a7b6-3c38aaafe79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:07.317143 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1f6d5-99cf-4815-a7b6-3c38aaafe79a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:07.317422 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1f6d5-99cf-4815-a7b6-3c38aaafe79a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:07.317794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1f6d5-99cf-4815-a7b6-3c38aaafe79a None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:07.318424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1f6d5-99cf-4815-a7b6-3c38aaafe79a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:07.318726 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1f6d5-99cf-4815-a7b6-3c38aaafe79a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:07.322108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1f6d5-99cf-4815-a7b6-3c38aaafe79a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:07.336016 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 385/762] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:07.342490 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1f6d5-99cf-4815-a7b6-3c38aaafe79a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:07.448310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76939f21-48bc-45f2-967d-1298e2d223f5 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:07.455075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:07.466374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1f6d5-99cf-4815-a7b6-3c38aaafe79a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:07.490067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573747a-8305-4526-bba9-394237d2f19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:07.490910 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573747a-8305-4526-bba9-394237d2f19a None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:07.491246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573747a-8305-4526-bba9-394237d2f19a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:07.491561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573747a-8305-4526-bba9-394237d2f19a None admin] PATH_INFO: `/v3/projects/59f00389bb404b4392d98e22e045c84f/users/850a50cfe4394febbafb0908b08888b2/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:07.494072 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573747a-8305-4526-bba9-394237d2f19a None admin] RBAC: Authorizing `identity:create_grant(project_id=59f00389bb404b4392d98e22e045c84f, user_id=850a50cfe4394febbafb0908b08888b2, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:07.495687 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573747a-8305-4526-bba9-394237d2f19a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:07.507363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06fcb8-e1e4-4901-a9a6-e7b1719d330a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:07.508185 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06fcb8-e1e4-4901-a9a6-e7b1719d330a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:07.508495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06fcb8-e1e4-4901-a9a6-e7b1719d330a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:07.508773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06fcb8-e1e4-4901-a9a6-e7b1719d330a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:07.510892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 378/763] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:38:07 2022] PUT /identity/v3/projects/59f00389bb404b4392d98e22e045c84f/users/850a50cfe4394febbafb0908b08888b2/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:38:07.512374 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b69a8-96aa-4a99-be2f-deb8f84e1c4e None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:07.512667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b69a8-96aa-4a99-be2f-deb8f84e1c4e None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:07.513000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b69a8-96aa-4a99-be2f-deb8f84e1c4e None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:07.514399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 379/764] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:07.516608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06fcb8-e1e4-4901-a9a6-e7b1719d330a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:07.518162 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06fcb8-e1e4-4901-a9a6-e7b1719d330a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:07.520496 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a59c6-d0cd-4b90-a91b-507b048a573c None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:07.520839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a59c6-d0cd-4b90-a91b-507b048a573c None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:07.521124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a59c6-d0cd-4b90-a91b-507b048a573c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:07.532578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-7e06fcb8-e1e4-4901-a9a6-e7b1719d330a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:07.532578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:07.532578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:07.532578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:07.532578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:07.532578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:07.532578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:07.532578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:07.532578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:07.532578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:07.532578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:07.532578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:07.532578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:07.532578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:07.532578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:07.533571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:07.534483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:07.534483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:07.534483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:07.534483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:07.534483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:07.534483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:07.534483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:07.534483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:07.534483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 386/765] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:07.545165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06fcb8-e1e4-4901-a9a6-e7b1719d330a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:07.550024 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-db7a59c6-d0cd-4b90-a91b-507b048a573c None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:07.551474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-db7a59c6-d0cd-4b90-a91b-507b048a573c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:07.664101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:07.687629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 380/766] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:07.734476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06fcb8-e1e4-4901-a9a6-e7b1719d330a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:07.736007 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a59c6-d0cd-4b90-a91b-507b048a573c None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:07.793482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4423536-e5bf-4444-b30f-e0330a8275db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:07.794112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4423536-e5bf-4444-b30f-e0330a8275db None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:07.794398 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4423536-e5bf-4444-b30f-e0330a8275db None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:07.794679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4423536-e5bf-4444-b30f-e0330a8275db None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:07.795458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4423536-e5bf-4444-b30f-e0330a8275db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:07.795771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4423536-e5bf-4444-b30f-e0330a8275db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:07.797288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4423536-e5bf-4444-b30f-e0330a8275db None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:07.814025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 387/767] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:38:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:07.819923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4423536-e5bf-4444-b30f-e0330a8275db None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:07.836086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:07.863777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a59c6-d0cd-4b90-a91b-507b048a573c None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:07.902647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fc7d2-5ee2-44f0-a523-c618549d2b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:07.903301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fc7d2-5ee2-44f0-a523-c618549d2b47 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:07.903642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fc7d2-5ee2-44f0-a523-c618549d2b47 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:07.903924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fc7d2-5ee2-44f0-a523-c618549d2b47 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:07.904607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fc7d2-5ee2-44f0-a523-c618549d2b47 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:07.906039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fc7d2-5ee2-44f0-a523-c618549d2b47 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-a39fc7d2-5ee2-44f0-a523-c618549d2b47 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:07.916358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:07.917365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:07.918203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 381/768] 10.222.0.69 () {56 vars in 1129 bytes} [Wed Aug 3 18:38:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:07.925536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fc7d2-5ee2-44f0-a523-c618549d2b47 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:07.942129 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4423536-e5bf-4444-b30f-e0330a8275db None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:07.984476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608f62c-68ba-421f-b845-4f8302e866b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:07.985119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608f62c-68ba-421f-b845-4f8302e866b5 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:07.985401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608f62c-68ba-421f-b845-4f8302e866b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:07.985675 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608f62c-68ba-421f-b845-4f8302e866b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:07.988983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608f62c-68ba-421f-b845-4f8302e866b5 None admin] RBAC: Adding query filter params (role.id=d67cdb8bae3648fd8f3589ab33176d14, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=6cebfb2bb96942c9b64720d9b8d9c6b2) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:07.989303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608f62c-68ba-421f-b845-4f8302e866b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:07.993985 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608f62c-68ba-421f-b845-4f8302e866b5 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:08.006357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 388/769] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:38:07 2022] GET /identity/v3/role_assignments?user.id=6cebfb2bb96942c9b64720d9b8d9c6b2&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=d67cdb8bae3648fd8f3589ab33176d14 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:08.037438 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:08.110539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fc7d2-5ee2-44f0-a523-c618549d2b47 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:08.160810 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11bb4f8-4e02-4eab-9ce3-3f255406981a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:08.161452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bb4f8-4e02-4eab-9ce3-3f255406981a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:08.161739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bb4f8-4e02-4eab-9ce3-3f255406981a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:08.162021 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bb4f8-4e02-4eab-9ce3-3f255406981a None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:08.162721 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11bb4f8-4e02-4eab-9ce3-3f255406981a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:08.163029 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11bb4f8-4e02-4eab-9ce3-3f255406981a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:08.165131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11bb4f8-4e02-4eab-9ce3-3f255406981a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:08.169978 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 382/770] 10.222.0.69 () {56 vars in 1124 bytes} [Wed Aug 3 18:38:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:08.175831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608f62c-68ba-421f-b845-4f8302e866b5 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:08.229704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3b2b1-695e-4688-a5a8-2709a052ba31 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:08.230050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3b2b1-695e-4688-a5a8-2709a052ba31 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:08.230686 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3b2b1-695e-4688-a5a8-2709a052ba31 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:08.232210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 383/771] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:08.237222 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3e6f9-6a8f-475a-9e21-2d5ed80c48f5 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:08.237511 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3e6f9-6a8f-475a-9e21-2d5ed80c48f5 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:08.237787 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3e6f9-6a8f-475a-9e21-2d5ed80c48f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:08.266713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-72d3e6f9-6a8f-475a-9e21-2d5ed80c48f5 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:08.267905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-72d3e6f9-6a8f-475a-9e21-2d5ed80c48f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:08.290966 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:08.331802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608f62c-68ba-421f-b845-4f8302e866b5 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:08.408827 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 384/772] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:08.409991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100ae44-2e4d-49a3-b58e-7567606f119a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:08.410632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100ae44-2e4d-49a3-b58e-7567606f119a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:08.410914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100ae44-2e4d-49a3-b58e-7567606f119a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:08.411189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100ae44-2e4d-49a3-b58e-7567606f119a None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:08.411885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100ae44-2e4d-49a3-b58e-7567606f119a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:08.412189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100ae44-2e4d-49a3-b58e-7567606f119a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:08.413732 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100ae44-2e4d-49a3-b58e-7567606f119a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:08.424780 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 389/773] 10.222.0.69 () {56 vars in 1124 bytes} [Wed Aug 3 18:38:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:08.431073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3e6f9-6a8f-475a-9e21-2d5ed80c48f5 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:08.460789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100ae44-2e4d-49a3-b58e-7567606f119a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:08.515390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:08.626656 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3e6f9-6a8f-475a-9e21-2d5ed80c48f5 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:08.633923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100ae44-2e4d-49a3-b58e-7567606f119a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:08.676621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b635d2f-63cc-4091-963b-06e78b205a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:08.677227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b635d2f-63cc-4091-963b-06e78b205a8e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:08.677512 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b635d2f-63cc-4091-963b-06e78b205a8e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:08.677784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b635d2f-63cc-4091-963b-06e78b205a8e None admin] PATH_INFO: `/v3/services/network` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:08.678415 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b635d2f-63cc-4091-963b-06e78b205a8e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:08.680984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b635d2f-63cc-4091-963b-06e78b205a8e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-9b635d2f-63cc-4091-963b-06e78b205a8e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:08.692116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:08.693315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:08.694166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:08.694166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:08.694166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:08.694166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:08.694166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:08.694166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:08.694166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:08.694166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:08.694166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:08.694166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:08.694166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:08.694166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:08.694166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 18:38:08.694166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:08.694166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 390/774] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:08 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:08.701202 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b635d2f-63cc-4091-963b-06e78b205a8e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:08.744851 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:08.749010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f98ba-d5b3-4198-b04a-6f1aa6cb2445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:08.749851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f98ba-d5b3-4198-b04a-6f1aa6cb2445 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:08.750142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f98ba-d5b3-4198-b04a-6f1aa6cb2445 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:08.750425 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f98ba-d5b3-4198-b04a-6f1aa6cb2445 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:08.751222 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f98ba-d5b3-4198-b04a-6f1aa6cb2445 None admin] RBAC: Adding query filter params (interface=public, service_id=5ede3cb7400b461191d4a726b5c79e5d, region_id=RegionOne) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:08.751541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f98ba-d5b3-4198-b04a-6f1aa6cb2445 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:08.753209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f98ba-d5b3-4198-b04a-6f1aa6cb2445 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:08.778365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 385/775] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Aug 3 18:38:08 2022] GET /identity/v3/endpoints?service_id=5ede3cb7400b461191d4a726b5c79e5d&interface=public®ion_id=RegionOne => generated 194 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:08.797903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f98ba-d5b3-4198-b04a-6f1aa6cb2445 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:08.837955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b635d2f-63cc-4091-963b-06e78b205a8e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:08.892961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3d5a6-ccd1-4a4f-9ee6-e35e999157cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:08.893897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3d5a6-ccd1-4a4f-9ee6-e35e999157cb None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:08.894188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3d5a6-ccd1-4a4f-9ee6-e35e999157cb None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:08.894468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3d5a6-ccd1-4a4f-9ee6-e35e999157cb None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:08.895818 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3d5a6-ccd1-4a4f-9ee6-e35e999157cb None admin] RBAC: Adding query filter params (name=network) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:08.896249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3d5a6-ccd1-4a4f-9ee6-e35e999157cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:08.899836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3d5a6-ccd1-4a4f-9ee6-e35e999157cb None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:08.908515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 391/776] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Aug 3 18:38:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:08.918602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3d5a6-ccd1-4a4f-9ee6-e35e999157cb None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:09.008314 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:09.054551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3d5a6-ccd1-4a4f-9ee6-e35e999157cb None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:09.099630 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8bc85-3fac-4734-90cd-c861736b7554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:09.100456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8bc85-3fac-4734-90cd-c861736b7554 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:09.101507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8bc85-3fac-4734-90cd-c861736b7554 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:09.102106 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8bc85-3fac-4734-90cd-c861736b7554 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:09.103086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8bc85-3fac-4734-90cd-c861736b7554 None admin] RBAC: Adding query filter params (type=network) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:09.103672 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8bc85-3fac-4734-90cd-c861736b7554 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:09.105810 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8bc85-3fac-4734-90cd-c861736b7554 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:09.117979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 392/777] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Aug 3 18:38:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:09.125317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8bc85-3fac-4734-90cd-c861736b7554 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:09.135168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f98ba-d5b3-4198-b04a-6f1aa6cb2445 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:09.188659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:09.226549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603b9be-296b-40e2-9a2e-76d774fd9850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:09.227608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603b9be-296b-40e2-9a2e-76d774fd9850 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:09.228071 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603b9be-296b-40e2-9a2e-76d774fd9850 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:09.232724 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603b9be-296b-40e2-9a2e-76d774fd9850 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:09.233683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603b9be-296b-40e2-9a2e-76d774fd9850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:09.235241 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603b9be-296b-40e2-9a2e-76d774fd9850 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:09.246722 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 386/778] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Aug 3 18:38:08 2022] GET /identity/v3/services => generated 772 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:09.260326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8bc85-3fac-4734-90cd-c861736b7554 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:09.312597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:09.332335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97e2cf-65c4-4746-a682-09e62c880687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:09.333574 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97e2cf-65c4-4746-a682-09e62c880687 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:09.334052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97e2cf-65c4-4746-a682-09e62c880687 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:09.334687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97e2cf-65c4-4746-a682-09e62c880687 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:09.336505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97e2cf-65c4-4746-a682-09e62c880687 None admin] RBAC: Adding query filter params (interface=public, service_id=a58bf17968cc4067a75555e9155c0c79, region_id=RegionOne) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:09.337266 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97e2cf-65c4-4746-a682-09e62c880687 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:09.338882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97e2cf-65c4-4746-a682-09e62c880687 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:09.360406 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 393/779] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Aug 3 18:38:09 2022] GET /identity/v3/endpoints?service_id=a58bf17968cc4067a75555e9155c0c79&interface=public®ion_id=RegionOne => generated 194 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:09.372897 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603b9be-296b-40e2-9a2e-76d774fd9850 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:09.521673 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603b9be-296b-40e2-9a2e-76d774fd9850 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:09.566558 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dc482-e254-45df-97e0-a87d44e6dde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:09.567546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dc482-e254-45df-97e0-a87d44e6dde1 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:09.568025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dc482-e254-45df-97e0-a87d44e6dde1 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:09.568516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dc482-e254-45df-97e0-a87d44e6dde1 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:09.569373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6dc482-e254-45df-97e0-a87d44e6dde1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:09.571600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6dc482-e254-45df-97e0-a87d44e6dde1 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:09.580826 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 387/780] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Aug 3 18:38:09 2022] GET /identity/v3/services => generated 772 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:09.791624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732363a6-629a-4a50-8e2e-9ebc20b698f6 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:09.791624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732363a6-629a-4a50-8e2e-9ebc20b698f6 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:09.791624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732363a6-629a-4a50-8e2e-9ebc20b698f6 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:09.792395 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 394/781] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:09.798195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd25a2-30f4-4172-9852-ea9ad3d0515f None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:09.798778 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd25a2-30f4-4172-9852-ea9ad3d0515f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:09.799261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd25a2-30f4-4172-9852-ea9ad3d0515f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:09.830057 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-dacd25a2-30f4-4172-9852-ea9ad3d0515f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:09.831649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-dacd25a2-30f4-4172-9852-ea9ad3d0515f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:09.888874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:09.942641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 388/782] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:09.978559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732363a6-629a-4a50-8e2e-9ebc20b698f6 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:10.099640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732363a6-629a-4a50-8e2e-9ebc20b698f6 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:10.139307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744f911-90f4-4894-a581-fe5410434ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:10.139959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744f911-90f4-4894-a581-fe5410434ce7 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:10.140316 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744f911-90f4-4894-a581-fe5410434ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:10.140607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744f911-90f4-4894-a581-fe5410434ce7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:10.153868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744f911-90f4-4894-a581-fe5410434ce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-9744f911-90f4-4894-a581-fe5410434ce7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:10.161462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:10.162392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:10.163327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:10.163327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:10.163327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:10.163327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:10.163327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:10.163327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:10.163327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:10.163327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:10.163327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:10.163327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:10.163327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:10.163327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:10.163327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 395/783] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:38:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:10.169561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd25a2-30f4-4172-9852-ea9ad3d0515f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:10.295639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd25a2-30f4-4172-9852-ea9ad3d0515f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:10.331973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928da15-a640-4813-948f-3fd9fb758cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:10.332631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928da15-a640-4813-948f-3fd9fb758cbe None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:10.332915 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928da15-a640-4813-948f-3fd9fb758cbe None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:10.333193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928da15-a640-4813-948f-3fd9fb758cbe None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:10.333934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d928da15-a640-4813-948f-3fd9fb758cbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:10.334239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d928da15-a640-4813-948f-3fd9fb758cbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:10.335929 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d928da15-a640-4813-948f-3fd9fb758cbe None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:10.345386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 389/784] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:38:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:10.351626 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744f911-90f4-4894-a581-fe5410434ce7 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:10.371753 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dc7b9-b584-4260-94cc-48960e2e6147 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:10.372053 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dc7b9-b584-4260-94cc-48960e2e6147 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:10.372315 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dc7b9-b584-4260-94cc-48960e2e6147 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:10.373953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 390/785] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:38:10.380275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50347e2-fb75-445d-8e0b-4c36f614732e None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:10.380583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50347e2-fb75-445d-8e0b-4c36f614732e None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:10.380859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50347e2-fb75-445d-8e0b-4c36f614732e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:10.405590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-b50347e2-fb75-445d-8e0b-4c36f614732e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:10.407040 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b50347e2-fb75-445d-8e0b-4c36f614732e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:10.440873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:10.465889 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:10.519562 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 391/786] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:10.542051 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744f911-90f4-4894-a581-fe5410434ce7 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:10.548113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50347e2-fb75-445d-8e0b-4c36f614732e None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:10.592979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa038b-290d-4734-a4c9-aad9ff2b8765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:10.593634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa038b-290d-4734-a4c9-aad9ff2b8765 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:10.593959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa038b-290d-4734-a4c9-aad9ff2b8765 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:10.594247 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa038b-290d-4734-a4c9-aad9ff2b8765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:10.606700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa038b-290d-4734-a4c9-aad9ff2b8765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:10.608240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa038b-290d-4734-a4c9-aad9ff2b8765 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:10.628784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-4baa038b-290d-4734-a4c9-aad9ff2b8765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:10.628784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:10.628784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:10.628784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:10.628784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:10.628784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:10.628784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:10.628784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:10.628784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:10.628784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:10.628784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:10.628784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:10.628784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:10.628784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:10.628784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:10.630923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:10.633776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:10.633776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:10.633776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:10.633776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:10.633776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 396/787] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:10.634712 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc01d1-be6f-4f75-9bcb-b30546b7bc40 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:10.634712 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc01d1-be6f-4f75-9bcb-b30546b7bc40 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:10.636329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc01d1-be6f-4f75-9bcb-b30546b7bc40 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:10.640360 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 397/788] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:10.641318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc01d1-be6f-4f75-9bcb-b30546b7bc40 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:10.673369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50347e2-fb75-445d-8e0b-4c36f614732e None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:10.711002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94461d12-bf1f-4590-9546-6e5cb9dcea85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:10.711716 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94461d12-bf1f-4590-9546-6e5cb9dcea85 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:10.712002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94461d12-bf1f-4590-9546-6e5cb9dcea85 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:10.712277 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94461d12-bf1f-4590-9546-6e5cb9dcea85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:10.723114 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94461d12-bf1f-4590-9546-6e5cb9dcea85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-94461d12-bf1f-4590-9546-6e5cb9dcea85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:10.728272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:10.729448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:10.730497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:10.730497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:10.730497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:10.730497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:10.730497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:10.730497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:10.730497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:10.730497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:10.730497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:10.730497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:10.730497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:10.730497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:10.732894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 392/789] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:38:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:10.734295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebab670-fcb0-428c-9d4c-b8f7eef52552 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:10.734709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebab670-fcb0-428c-9d4c-b8f7eef52552 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:10.735113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebab670-fcb0-428c-9d4c-b8f7eef52552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:10.755208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-3ebab670-fcb0-428c-9d4c-b8f7eef52552 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:10.757018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-3ebab670-fcb0-428c-9d4c-b8f7eef52552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:10.758341 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc01d1-be6f-4f75-9bcb-b30546b7bc40 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:10.769290 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:10.777820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:10.797382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5daaa0-a3fd-4f33-a4f5-a49c14d46269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:10.798000 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5daaa0-a3fd-4f33-a4f5-a49c14d46269 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:10.798304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5daaa0-a3fd-4f33-a4f5-a49c14d46269 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:10.798572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5daaa0-a3fd-4f33-a4f5-a49c14d46269 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:10.799603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5daaa0-a3fd-4f33-a4f5-a49c14d46269 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:10.799603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5daaa0-a3fd-4f33-a4f5-a49c14d46269 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:10.800962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5daaa0-a3fd-4f33-a4f5-a49c14d46269 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:10.818452 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 398/790] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:10.819375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5daaa0-a3fd-4f33-a4f5-a49c14d46269 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:10.844390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 393/791] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:10.845631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebab670-fcb0-428c-9d4c-b8f7eef52552 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:10.929709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5daaa0-a3fd-4f33-a4f5-a49c14d46269 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:10.966880 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd118e5-eeb5-453a-a4b4-8e6bb65f558e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:10.967515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd118e5-eeb5-453a-a4b4-8e6bb65f558e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:10.967799 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd118e5-eeb5-453a-a4b4-8e6bb65f558e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:10.968105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd118e5-eeb5-453a-a4b4-8e6bb65f558e None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:10.968847 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd118e5-eeb5-453a-a4b4-8e6bb65f558e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:10.969163 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd118e5-eeb5-453a-a4b4-8e6bb65f558e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:10.970527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd118e5-eeb5-453a-a4b4-8e6bb65f558e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:10.975404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebab670-fcb0-428c-9d4c-b8f7eef52552 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:10.979525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 399/792] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:38:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:10.980457 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd118e5-eeb5-453a-a4b4-8e6bb65f558e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:11.013113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:11.015832 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed139c-444c-4f47-9daf-eebc506cba7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:11.016839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed139c-444c-4f47-9daf-eebc506cba7e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:11.017290 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed139c-444c-4f47-9daf-eebc506cba7e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:11.017709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed139c-444c-4f47-9daf-eebc506cba7e None admin] PATH_INFO: `/v3/users/swift` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:11.027484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed139c-444c-4f47-9daf-eebc506cba7e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:11.029195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed139c-444c-4f47-9daf-eebc506cba7e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-e4ed139c-444c-4f47-9daf-eebc506cba7e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:38:11.041628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:11.042760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:11.043851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:11.043851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:11.043851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:38:11.043851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:38:11.043851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:38:11.043851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:38:11.043851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:38:11.043851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:38:11.043851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 18:38:11.043851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:11.045947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 394/793] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:38:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:11.046864 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed139c-444c-4f47-9daf-eebc506cba7e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:11.072001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:11.101388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd118e5-eeb5-453a-a4b4-8e6bb65f558e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:11.151211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30fdc1-2c1c-4c8f-8911-da4a0ac9693a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:11.152113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30fdc1-2c1c-4c8f-8911-da4a0ac9693a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:11.152968 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30fdc1-2c1c-4c8f-8911-da4a0ac9693a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:11.153661 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30fdc1-2c1c-4c8f-8911-da4a0ac9693a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:11.167976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30fdc1-2c1c-4c8f-8911-da4a0ac9693a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:11.175425 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed139c-444c-4f47-9daf-eebc506cba7e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-6b30fdc1-2c1c-4c8f-8911-da4a0ac9693a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:11.175720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:11.176885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:11.177909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:11.177909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:11.177909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:11.177909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:11.177909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:11.177909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:11.177909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:11.177909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:11.177909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:11.177909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:11.177909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:38:11.177909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:11.187163 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 400/794] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:38:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:11.188324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30fdc1-2c1c-4c8f-8911-da4a0ac9693a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:11.223584 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8eda5-18d7-472f-8f75-f661c574d763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:11.224897 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8eda5-18d7-472f-8f75-f661c574d763 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:11.225186 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8eda5-18d7-472f-8f75-f661c574d763 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:11.226326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8eda5-18d7-472f-8f75-f661c574d763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:11.240831 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8eda5-18d7-472f-8f75-f661c574d763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:11.243423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8eda5-18d7-472f-8f75-f661c574d763 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:11.264800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-03e8eda5-18d7-472f-8f75-f661c574d763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:11.264800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:11.264800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:11.264800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:11.264800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:11.264800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:11.264800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:11.264800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:11.264800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:11.264800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:11.264800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:11.264800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:11.264800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:11.264800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:11.264800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:11.265786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:11.266693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:11.266693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:11.266693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:11.266693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:11.266693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 395/795] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:11.268333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8eda5-18d7-472f-8f75-f661c574d763 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:11.301391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:11.313560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30fdc1-2c1c-4c8f-8911-da4a0ac9693a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:11.352070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:11.363151 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df2699-ec6a-4377-8870-bd20bb131364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:11.363803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df2699-ec6a-4377-8870-bd20bb131364 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:11.364057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df2699-ec6a-4377-8870-bd20bb131364 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:11.364350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df2699-ec6a-4377-8870-bd20bb131364 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:11.365162 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df2699-ec6a-4377-8870-bd20bb131364 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:11.365469 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df2699-ec6a-4377-8870-bd20bb131364 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:11.366950 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df2699-ec6a-4377-8870-bd20bb131364 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:11.391310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 401/796] 10.222.0.69 () {56 vars in 1135 bytes} [Wed Aug 3 18:38:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:11.392158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df2699-ec6a-4377-8870-bd20bb131364 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:11.396711 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8eda5-18d7-472f-8f75-f661c574d763 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:11.439488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008949ec-596a-41d8-8dd7-f60b7cae6895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:11.440035 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008949ec-596a-41d8-8dd7-f60b7cae6895 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:11.440332 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008949ec-596a-41d8-8dd7-f60b7cae6895 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:11.440614 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008949ec-596a-41d8-8dd7-f60b7cae6895 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:11.441330 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008949ec-596a-41d8-8dd7-f60b7cae6895 None admin] RBAC: Adding query filter params (name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:11.441639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008949ec-596a-41d8-8dd7-f60b7cae6895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:11.443446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008949ec-596a-41d8-8dd7-f60b7cae6895 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:11.456372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 396/797] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:38:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:11.457632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008949ec-596a-41d8-8dd7-f60b7cae6895 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:11.522265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df2699-ec6a-4377-8870-bd20bb131364 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:11.567952 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940552c-2000-4e1d-9fb3-f00673ad172d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:11.568860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940552c-2000-4e1d-9fb3-f00673ad172d None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:11.569145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940552c-2000-4e1d-9fb3-f00673ad172d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:11.569945 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940552c-2000-4e1d-9fb3-f00673ad172d None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:11.570705 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940552c-2000-4e1d-9fb3-f00673ad172d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:11.571017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940552c-2000-4e1d-9fb3-f00673ad172d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:11.573107 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940552c-2000-4e1d-9fb3-f00673ad172d None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:11.576056 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008949ec-596a-41d8-8dd7-f60b7cae6895 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:11.587581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 402/798] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:11.588899 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940552c-2000-4e1d-9fb3-f00673ad172d None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:11.600778 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:11.609017 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:11.619463 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7865712-5e07-468a-bf16-4e92f76d00c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:11.619780 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7865712-5e07-468a-bf16-4e92f76d00c6 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:11.619977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7865712-5e07-468a-bf16-4e92f76d00c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:11.620237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7865712-5e07-468a-bf16-4e92f76d00c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:11.635902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7865712-5e07-468a-bf16-4e92f76d00c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:11.637465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7865712-5e07-468a-bf16-4e92f76d00c6 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:11.656209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-f7865712-5e07-468a-bf16-4e92f76d00c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:11.656209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:11.656209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:11.656209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:11.656209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:11.656209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:11.656209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:11.656209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:11.656209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:11.656209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:11.656209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:11.656209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:11.656209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:11.656209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:11.656209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:11.657185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:11.658115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:11.658115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:11.658115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:11.658115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:11.658115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 397/799] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:11.659788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7865712-5e07-468a-bf16-4e92f76d00c6 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:11.718947 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940552c-2000-4e1d-9fb3-f00673ad172d None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:11.756701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53107b92-c6ab-412d-8265-12830a0c3d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:11.757947 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53107b92-c6ab-412d-8265-12830a0c3d34 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:11.758253 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53107b92-c6ab-412d-8265-12830a0c3d34 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:11.758584 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53107b92-c6ab-412d-8265-12830a0c3d34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:11.770318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53107b92-c6ab-412d-8265-12830a0c3d34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:11.771783 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53107b92-c6ab-412d-8265-12830a0c3d34 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:11.780769 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7865712-5e07-468a-bf16-4e92f76d00c6 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:11.788004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-53107b92-c6ab-412d-8265-12830a0c3d34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:11.788004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:11.788004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:11.788004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:11.788004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:11.788004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:11.788004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:11.788004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:11.788004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:11.788004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:11.788004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:11.788004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:11.788004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:11.788004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:11.788004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:11.788992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:11.789921 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:11.789921 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:11.789921 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:11.789921 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:11.794264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 403/800] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:11.795009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53107b92-c6ab-412d-8265-12830a0c3d34 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:11.822750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af2231-c675-45b5-996d-5ea34033e861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:11.823406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af2231-c675-45b5-996d-5ea34033e861 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:11.823696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af2231-c675-45b5-996d-5ea34033e861 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:11.823976 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af2231-c675-45b5-996d-5ea34033e861 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:11.832402 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af2231-c675-45b5-996d-5ea34033e861 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:11.833845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af2231-c675-45b5-996d-5ea34033e861 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-c1af2231-c675-45b5-996d-5ea34033e861 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:38:11.848245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:11.849409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:11.850473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:11.850473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:11.850473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:38:11.850473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:38:11.850473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:38:11.850473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:38:11.850473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:38:11.850473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:38:11.850473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:38:11.850473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:11.853280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 398/801] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Aug 3 18:38:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:11.854223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af2231-c675-45b5-996d-5ea34033e861 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:11.870338 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:11.904934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:11.921914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53107b92-c6ab-412d-8265-12830a0c3d34 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:11.961942 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3ae1b-215b-4093-8a09-cd6044b64b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:11.962567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3ae1b-215b-4093-8a09-cd6044b64b56 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:11.962857 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3ae1b-215b-4093-8a09-cd6044b64b56 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:11.963125 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3ae1b-215b-4093-8a09-cd6044b64b56 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:11.963831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3ae1b-215b-4093-8a09-cd6044b64b56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:11.964135 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3ae1b-215b-4093-8a09-cd6044b64b56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:11.965537 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3ae1b-215b-4093-8a09-cd6044b64b56 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:11.976266 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af2231-c675-45b5-996d-5ea34033e861 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:11.984190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 404/802] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:11.985649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798580c1-fcc5-4aac-a06f-cb97452d9cec None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:11.985936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798580c1-fcc5-4aac-a06f-cb97452d9cec None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:11.986212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798580c1-fcc5-4aac-a06f-cb97452d9cec None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:11.987487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 405/803] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:11.988411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798580c1-fcc5-4aac-a06f-cb97452d9cec None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:12.018900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c36ab-da60-426b-bf85-bea7c60e4b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:12.019527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c36ab-da60-426b-bf85-bea7c60e4b74 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:12.019804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c36ab-da60-426b-bf85-bea7c60e4b74 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:12.020077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c36ab-da60-426b-bf85-bea7c60e4b74 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:12.020807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c36ab-da60-426b-bf85-bea7c60e4b74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:12.021098 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c36ab-da60-426b-bf85-bea7c60e4b74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:12.022984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c36ab-da60-426b-bf85-bea7c60e4b74 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:12.037641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 399/804] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:12.038940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c36ab-da60-426b-bf85-bea7c60e4b74 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:12.103349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798580c1-fcc5-4aac-a06f-cb97452d9cec None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:12.124502 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:12.141333 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097009ae-dfc3-4471-9cd7-33df06bf0f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:12.141955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097009ae-dfc3-4471-9cd7-33df06bf0f3a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:12.142246 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097009ae-dfc3-4471-9cd7-33df06bf0f3a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:12.142515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097009ae-dfc3-4471-9cd7-33df06bf0f3a None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:12.143105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:12.143318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097009ae-dfc3-4471-9cd7-33df06bf0f3a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:12.143625 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097009ae-dfc3-4471-9cd7-33df06bf0f3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:12.145099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097009ae-dfc3-4471-9cd7-33df06bf0f3a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:12.147897 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c36ab-da60-426b-bf85-bea7c60e4b74 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:12.168691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 406/805] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Aug 3 18:38:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:12.168691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782755c3-94d1-4b35-a4f8-844d9a99d7a5 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:12.168691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782755c3-94d1-4b35-a4f8-844d9a99d7a5 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:12.168691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782755c3-94d1-4b35-a4f8-844d9a99d7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:12.185082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-782755c3-94d1-4b35-a4f8-844d9a99d7a5 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:12.186270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-782755c3-94d1-4b35-a4f8-844d9a99d7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:12.192685 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1a827-2909-4864-88b0-d0cde8dfb957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:12.193348 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1a827-2909-4864-88b0-d0cde8dfb957 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:12.193657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1a827-2909-4864-88b0-d0cde8dfb957 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:12.193949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1a827-2909-4864-88b0-d0cde8dfb957 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:12.202028 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1a827-2909-4864-88b0-d0cde8dfb957 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:12.203496 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1a827-2909-4864-88b0-d0cde8dfb957 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:12.215127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-07d1a827-2909-4864-88b0-d0cde8dfb957 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:12.215127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:12.215127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:12.215127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:12.215127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:12.215127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:12.215127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:12.215127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:12.215127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:12.215127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:12.215127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:12.215127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:12.215127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:12.215127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:12.215127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:12.216050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:12.217337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:12.217337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:12.217337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:12.217337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:12.217337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:12.217337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:12.217337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:12.217337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:12.217337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 400/806] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:12.217337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1a827-2909-4864-88b0-d0cde8dfb957 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:12.263250 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 407/807] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:12.264007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782755c3-94d1-4b35-a4f8-844d9a99d7a5 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:12.321196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1a827-2909-4864-88b0-d0cde8dfb957 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:12.354750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1792a7-0147-4162-af19-35f39199e42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:12.355391 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1792a7-0147-4162-af19-35f39199e42a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:12.355676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1792a7-0147-4162-af19-35f39199e42a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:12.355950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1792a7-0147-4162-af19-35f39199e42a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:12.363504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1792a7-0147-4162-af19-35f39199e42a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:12.364970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1792a7-0147-4162-af19-35f39199e42a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:12.372225 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782755c3-94d1-4b35-a4f8-844d9a99d7a5 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-1a1792a7-0147-4162-af19-35f39199e42a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:38:12.376763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:12.378127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:12.379331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:12.379331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:12.379331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:38:12.379331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:38:12.379331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:38:12.379331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:38:12.379331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:38:12.379331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:38:12.379331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 18:38:12.379331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:12.380426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 401/808] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Aug 3 18:38:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:12.381404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1792a7-0147-4162-af19-35f39199e42a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:12.383084 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:12.403489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:12.412171 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8f41d-61bc-4af0-8da2-a6c6840c1d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:12.412814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8f41d-61bc-4af0-8da2-a6c6840c1d64 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:12.413761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8f41d-61bc-4af0-8da2-a6c6840c1d64 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:12.414295 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8f41d-61bc-4af0-8da2-a6c6840c1d64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:12.482136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1792a7-0147-4162-af19-35f39199e42a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:12.484560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8f41d-61bc-4af0-8da2-a6c6840c1d64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:12.486027 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8f41d-61bc-4af0-8da2-a6c6840c1d64 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:12.518274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0c48f-2ea2-4288-9193-94fcc25607d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:12.518809 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0c48f-2ea2-4288-9193-94fcc25607d9 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:12.519091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0c48f-2ea2-4288-9193-94fcc25607d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:12.519373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0c48f-2ea2-4288-9193-94fcc25607d9 None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:12.520120 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0c48f-2ea2-4288-9193-94fcc25607d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:12.520457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0c48f-2ea2-4288-9193-94fcc25607d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:12.521877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0c48f-2ea2-4288-9193-94fcc25607d9 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:12.534781 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 402/809] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:38:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:12.536070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdbfd5-05e1-4271-89bb-a37bec892dcd None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:12.536389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdbfd5-05e1-4271-89bb-a37bec892dcd None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:12.536671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdbfd5-05e1-4271-89bb-a37bec892dcd None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:12.538634 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 403/810] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:12.539135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdbfd5-05e1-4271-89bb-a37bec892dcd None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:12.592848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 408/811] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:38:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:38:12.593572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8f41d-61bc-4af0-8da2-a6c6840c1d64 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:12.604890 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:12.623190 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:12.643998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdbfd5-05e1-4271-89bb-a37bec892dcd None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:12.680395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4781cd-a153-4770-a4b5-8ebc3e103627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:12.681035 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4781cd-a153-4770-a4b5-8ebc3e103627 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:12.681319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4781cd-a153-4770-a4b5-8ebc3e103627 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:12.681598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4781cd-a153-4770-a4b5-8ebc3e103627 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:12.682251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4781cd-a153-4770-a4b5-8ebc3e103627 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:12.683702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4781cd-a153-4770-a4b5-8ebc3e103627 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-cb4781cd-a153-4770-a4b5-8ebc3e103627 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:12.696009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:12.697218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:12.698113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:12.698113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:12.698113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:12.698113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:12.698113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:12.698113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:12.698113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:12.698113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:12.698113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:12.698113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:12.698113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:12.698113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:12.698113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:12.698113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 18:38:12.698113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:12.698933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8f41d-61bc-4af0-8da2-a6c6840c1d64 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:12.699010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 404/812] 10.222.0.69 () {56 vars in 1129 bytes} [Wed Aug 3 18:38:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:12.699398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4781cd-a153-4770-a4b5-8ebc3e103627 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:12.738393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baa330-69d7-4f81-b345-89a8a8219788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:12.739012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baa330-69d7-4f81-b345-89a8a8219788 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:12.739291 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baa330-69d7-4f81-b345-89a8a8219788 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:12.739562 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baa330-69d7-4f81-b345-89a8a8219788 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:12.740383 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82baa330-69d7-4f81-b345-89a8a8219788 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:12.740688 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82baa330-69d7-4f81-b345-89a8a8219788 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:12.742571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82baa330-69d7-4f81-b345-89a8a8219788 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:12.762112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 409/813] 10.222.0.69 () {56 vars in 1149 bytes} [Wed Aug 3 18:38:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:12.763353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e5a05-191c-480c-b19b-8bf5c25fe63b None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:12.763636 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e5a05-191c-480c-b19b-8bf5c25fe63b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:12.763908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e5a05-191c-480c-b19b-8bf5c25fe63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:12.782456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-601e5a05-191c-480c-b19b-8bf5c25fe63b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:12.783556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-601e5a05-191c-480c-b19b-8bf5c25fe63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:12.813901 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4781cd-a153-4770-a4b5-8ebc3e103627 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:12.843900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:12.848970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cd4c2-293e-4604-8635-0cc01049caeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:12.849641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cd4c2-293e-4604-8635-0cc01049caeb None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:12.849925 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cd4c2-293e-4604-8635-0cc01049caeb None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:12.850195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cd4c2-293e-4604-8635-0cc01049caeb None admin] PATH_INFO: `/v3/projects/59f00389bb404b4392d98e22e045c84f/users/96ed059287274ca392c6abb893551094/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:12.852366 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 410/814] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:12.852929 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6cd4c2-293e-4604-8635-0cc01049caeb None admin] RBAC: Authorizing `identity:create_grant(project_id=59f00389bb404b4392d98e22e045c84f, user_id=96ed059287274ca392c6abb893551094, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:12.853511 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e5a05-191c-480c-b19b-8bf5c25fe63b None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:12.854359 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6cd4c2-293e-4604-8635-0cc01049caeb None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:12.878810 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 405/815] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:38:12 2022] PUT /identity/v3/projects/59f00389bb404b4392d98e22e045c84f/users/96ed059287274ca392c6abb893551094/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:38:12.878810 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cd4c2-293e-4604-8635-0cc01049caeb None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:12.881172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:12.989242 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e5a05-191c-480c-b19b-8bf5c25fe63b None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:12.999834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cd4c2-293e-4604-8635-0cc01049caeb None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:13.037022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57830af-1c8a-486e-9442-721413bb6397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:13.037712 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57830af-1c8a-486e-9442-721413bb6397 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:13.038133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57830af-1c8a-486e-9442-721413bb6397 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:13.038488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57830af-1c8a-486e-9442-721413bb6397 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:13.041823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d133e10-6ece-4066-8cf6-d5e8f8457d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:13.042524 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d133e10-6ece-4066-8cf6-d5e8f8457d1e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:13.042881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d133e10-6ece-4066-8cf6-d5e8f8457d1e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:13.043227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d133e10-6ece-4066-8cf6-d5e8f8457d1e None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:13.043987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d133e10-6ece-4066-8cf6-d5e8f8457d1e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:13.044408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d133e10-6ece-4066-8cf6-d5e8f8457d1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:13.045854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d133e10-6ece-4066-8cf6-d5e8f8457d1e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:13.050491 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57830af-1c8a-486e-9442-721413bb6397 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:13.051494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 406/816] 10.222.0.69 () {56 vars in 1124 bytes} [Wed Aug 3 18:38:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:13.052001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57830af-1c8a-486e-9442-721413bb6397 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:13.052370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d133e10-6ece-4066-8cf6-d5e8f8457d1e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:13.070116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-e57830af-1c8a-486e-9442-721413bb6397 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:13.070116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:13.070116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:13.070116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:13.070116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:13.070116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:13.070116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:13.070116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:13.070116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:13.070116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:13.070116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:13.070116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:13.070116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:13.070116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:13.070116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:13.071494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:13.072481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:13.072481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:13.072481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:13.072481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:13.072481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 411/817] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:13.073201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57830af-1c8a-486e-9442-721413bb6397 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:13.128768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:13.143617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:13.171814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d133e10-6ece-4066-8cf6-d5e8f8457d1e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:13.203854 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57830af-1c8a-486e-9442-721413bb6397 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:13.216247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf4585-cf79-4141-8cbc-21cdeb3a00b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:13.217162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf4585-cf79-4141-8cbc-21cdeb3a00b0 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:13.217587 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf4585-cf79-4141-8cbc-21cdeb3a00b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:13.218120 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf4585-cf79-4141-8cbc-21cdeb3a00b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:13.230078 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf4585-cf79-4141-8cbc-21cdeb3a00b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:13.231793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf4585-cf79-4141-8cbc-21cdeb3a00b0 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:13.243015 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7eeab-d19c-4c7e-8d32-a647508cfe2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:13.243663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7eeab-d19c-4c7e-8d32-a647508cfe2e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:13.243943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7eeab-d19c-4c7e-8d32-a647508cfe2e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:13.244215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7eeab-d19c-4c7e-8d32-a647508cfe2e None admin] PATH_INFO: `/v3/services/network` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:13.244892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7eeab-d19c-4c7e-8d32-a647508cfe2e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:13.246257 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7eeab-d19c-4c7e-8d32-a647508cfe2e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:13.246422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-73cf4585-cf79-4141-8cbc-21cdeb3a00b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:13.246422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:13.246422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:13.246422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:13.246422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:13.246422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:13.246422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:13.246422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:13.246422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:13.246422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:13.246422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:13.246422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:13.246422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:13.246422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:13.246422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:13.247287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:13.248187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:13.248187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:13.248187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:13.248187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:13.248187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 407/818] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:13.248508 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf4585-cf79-4141-8cbc-21cdeb3a00b0 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-66f7eeab-d19c-4c7e-8d32-a647508cfe2e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:13.259413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:13.260766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:13.261763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:13.261763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:13.261763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:13.261763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:13.261763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:13.261763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:13.261763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:13.261763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:13.261763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:13.261763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:13.261763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:13.261763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:13.261763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 18:38:13.261763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:13.265285 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 412/819] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:13 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:13.266201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7eeab-d19c-4c7e-8d32-a647508cfe2e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:13.372463 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf4585-cf79-4141-8cbc-21cdeb3a00b0 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:13.379153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7eeab-d19c-4c7e-8d32-a647508cfe2e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:13.394288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:13.395600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:13.416900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b605c-7235-4e4b-8e3a-9c4ce76521e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:13.417791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b605c-7235-4e4b-8e3a-9c4ce76521e2 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:13.418198 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b605c-7235-4e4b-8e3a-9c4ce76521e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:13.418600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b605c-7235-4e4b-8e3a-9c4ce76521e2 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:13.419004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84742ef3-e74b-4c99-86c0-93663d8921c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:13.419621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b605c-7235-4e4b-8e3a-9c4ce76521e2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:13.419819 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84742ef3-e74b-4c99-86c0-93663d8921c9 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:13.420076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b605c-7235-4e4b-8e3a-9c4ce76521e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:13.420282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84742ef3-e74b-4c99-86c0-93663d8921c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:13.420678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84742ef3-e74b-4c99-86c0-93663d8921c9 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:13.421532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84742ef3-e74b-4c99-86c0-93663d8921c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:13.421951 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84742ef3-e74b-4c99-86c0-93663d8921c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:13.422185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b605c-7235-4e4b-8e3a-9c4ce76521e2 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:13.423500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84742ef3-e74b-4c99-86c0-93663d8921c9 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:13.428252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 408/820] 10.222.0.69 () {56 vars in 1124 bytes} [Wed Aug 3 18:38:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:13.429884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b605c-7235-4e4b-8e3a-9c4ce76521e2 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:13.446059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 413/821] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:13.447661 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84742ef3-e74b-4c99-86c0-93663d8921c9 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:13.541747 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b605c-7235-4e4b-8e3a-9c4ce76521e2 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:13.570596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84742ef3-e74b-4c99-86c0-93663d8921c9 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:13.580705 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1d519-5cc2-40bf-ab16-c70513439aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:13.581439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1d519-5cc2-40bf-ab16-c70513439aab None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:13.582114 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1d519-5cc2-40bf-ab16-c70513439aab None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:13.582469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1d519-5cc2-40bf-ab16-c70513439aab None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:13.583248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1d519-5cc2-40bf-ab16-c70513439aab None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:13.583627 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1d519-5cc2-40bf-ab16-c70513439aab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:13.585621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1d519-5cc2-40bf-ab16-c70513439aab None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:13.602898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 409/822] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:13.603871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1d519-5cc2-40bf-ab16-c70513439aab None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:13.608131 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4eb96-53c0-49ca-922d-e6191da466df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:13.608131 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4eb96-53c0-49ca-922d-e6191da466df None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:13.608413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4eb96-53c0-49ca-922d-e6191da466df None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:13.608607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4eb96-53c0-49ca-922d-e6191da466df None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:13.609312 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4eb96-53c0-49ca-922d-e6191da466df None admin] RBAC: Adding query filter params (name=network) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:13.609620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4eb96-53c0-49ca-922d-e6191da466df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:13.610998 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4eb96-53c0-49ca-922d-e6191da466df None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:13.620883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 414/823] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Aug 3 18:38:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:13.620883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4eb96-53c0-49ca-922d-e6191da466df None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:13.659241 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:13.666948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:13.727242 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1d519-5cc2-40bf-ab16-c70513439aab None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:13.730561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4eb96-53c0-49ca-922d-e6191da466df None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:13.764696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1bebf-5905-4022-920b-7f1bae2718b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:13.765469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1bebf-5905-4022-920b-7f1bae2718b3 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:13.765895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1bebf-5905-4022-920b-7f1bae2718b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:13.766237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1bebf-5905-4022-920b-7f1bae2718b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:13.766957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed1bebf-5905-4022-920b-7f1bae2718b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:13.767541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb4dca-45f9-4f3f-934f-67f39f8f806a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:13.768190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb4dca-45f9-4f3f-934f-67f39f8f806a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:13.768401 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed1bebf-5905-4022-920b-7f1bae2718b3 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:13.768491 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb4dca-45f9-4f3f-934f-67f39f8f806a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:13.768772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb4dca-45f9-4f3f-934f-67f39f8f806a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:13.777130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb4dca-45f9-4f3f-934f-67f39f8f806a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:13.778771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb4dca-45f9-4f3f-934f-67f39f8f806a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:13.796501 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 410/824] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:38:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:38:13.796501 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1bebf-5905-4022-920b-7f1bae2718b3 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:13.796781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-f6eb4dca-45f9-4f3f-934f-67f39f8f806a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:13.796781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:13.796781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:13.796781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:13.796781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:13.796781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:13.796781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:13.796781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:13.796781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:13.796781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:13.796781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:13.796781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:13.796781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:13.796781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:13.796781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:13.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:13.805784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:13.805784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:13.805784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:13.805784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:13.805784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:13.805784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:13.805784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:13.805784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:13.805784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 415/825] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:13.805784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb4dca-45f9-4f3f-934f-67f39f8f806a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:13.927698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:13.951386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1bebf-5905-4022-920b-7f1bae2718b3 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:13.953778 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:13.964492 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb4dca-45f9-4f3f-934f-67f39f8f806a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:13.999303 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5af28-7a2e-4517-a036-f66c10d0670b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:13.999956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5af28-7a2e-4517-a036-f66c10d0670b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:14.000234 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5af28-7a2e-4517-a036-f66c10d0670b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:14.000547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5af28-7a2e-4517-a036-f66c10d0670b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:14.009561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5af28-7a2e-4517-a036-f66c10d0670b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:14.009747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba2262-bc9b-46bb-a380-41ac41a03555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:14.010310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba2262-bc9b-46bb-a380-41ac41a03555 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:14.010594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba2262-bc9b-46bb-a380-41ac41a03555 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:14.010871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba2262-bc9b-46bb-a380-41ac41a03555 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:14.011027 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5af28-7a2e-4517-a036-f66c10d0670b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:14.011565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba2262-bc9b-46bb-a380-41ac41a03555 None admin] RBAC: Adding query filter params (type=network) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:14.011876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba2262-bc9b-46bb-a380-41ac41a03555 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:14.013315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba2262-bc9b-46bb-a380-41ac41a03555 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:14.020738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 416/826] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Aug 3 18:38:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:14.021624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba2262-bc9b-46bb-a380-41ac41a03555 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:14.029336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-34c5af28-7a2e-4517-a036-f66c10d0670b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:14.029336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:14.029336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:14.029336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:14.029336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:14.029336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:14.029336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:14.029336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:14.029336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:14.029336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:14.029336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:14.029336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:14.029336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:14.029336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:14.029336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:14.030439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:14.031391 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:14.031391 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:14.031391 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:14.031391 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:14.031391 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:14.031391 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:14.031391 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:14.031391 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:14.032134 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 411/827] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:14.032783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5af28-7a2e-4517-a036-f66c10d0670b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:14.141408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5af28-7a2e-4517-a036-f66c10d0670b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:14.144411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba2262-bc9b-46bb-a380-41ac41a03555 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:14.178139 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba0b7c-368c-4887-be89-fd084adc1e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:14.178723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba0b7c-368c-4887-be89-fd084adc1e82 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:14.179009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba0b7c-368c-4887-be89-fd084adc1e82 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:14.179287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba0b7c-368c-4887-be89-fd084adc1e82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:14.180010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba0b7c-368c-4887-be89-fd084adc1e82 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:14.181413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba0b7c-368c-4887-be89-fd084adc1e82 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:14.183598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fbeb8-581e-4d4e-b279-a01d253c809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:14.184556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fbeb8-581e-4d4e-b279-a01d253c809e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:14.185106 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fbeb8-581e-4d4e-b279-a01d253c809e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:14.185617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fbeb8-581e-4d4e-b279-a01d253c809e None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:14.186567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fbeb8-581e-4d4e-b279-a01d253c809e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:14.187076 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fbeb8-581e-4d4e-b279-a01d253c809e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:14.188895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fbeb8-581e-4d4e-b279-a01d253c809e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:14.203634 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 412/828] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:38:14 2022] POST /identity/v3/endpoints => generated 337 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:38:14.208435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba0b7c-368c-4887-be89-fd084adc1e82 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:14.208610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 417/829] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:38:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:14.210576 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fbeb8-581e-4d4e-b279-a01d253c809e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:14.264683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:14.266131 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:14.420850 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba0b7c-368c-4887-be89-fd084adc1e82 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:14.421377 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fbeb8-581e-4d4e-b279-a01d253c809e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:14.459215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb391d9f-e5aa-481d-a8c4-faac23b75ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:14.460460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb391d9f-e5aa-481d-a8c4-faac23b75ae8 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:14.460747 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb391d9f-e5aa-481d-a8c4-faac23b75ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:14.461088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb391d9f-e5aa-481d-a8c4-faac23b75ae8 None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:14.461985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb391d9f-e5aa-481d-a8c4-faac23b75ae8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:14.462440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb391d9f-e5aa-481d-a8c4-faac23b75ae8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:14.462727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908b5c5-d3f7-470e-98c1-78f27e6991e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:14.463616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908b5c5-d3f7-470e-98c1-78f27e6991e0 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:14.464034 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb391d9f-e5aa-481d-a8c4-faac23b75ae8 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:14.464359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908b5c5-d3f7-470e-98c1-78f27e6991e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:14.464838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908b5c5-d3f7-470e-98c1-78f27e6991e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:14.465843 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908b5c5-d3f7-470e-98c1-78f27e6991e0 None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=850a50cfe4394febbafb0908b08888b2) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:14.466379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908b5c5-d3f7-470e-98c1-78f27e6991e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:14.468057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908b5c5-d3f7-470e-98c1-78f27e6991e0 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:14.475100 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 418/830] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:38:14 2022] GET /identity/v3/role_assignments?user.id=850a50cfe4394febbafb0908b08888b2&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=19557e941d6f475ba92cf862846f9872 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:14.477909 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 413/831] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:38:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:14.484481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908b5c5-d3f7-470e-98c1-78f27e6991e0 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:14.590309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:14.612667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908b5c5-d3f7-470e-98c1-78f27e6991e0 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:14.652223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094ede5-a29e-4dbc-a943-d62cbb00c658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:14.652942 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094ede5-a29e-4dbc-a943-d62cbb00c658 None admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:14.653304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094ede5-a29e-4dbc-a943-d62cbb00c658 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:14.653585 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094ede5-a29e-4dbc-a943-d62cbb00c658 None admin] PATH_INFO: `/v3/projects/59f00389bb404b4392d98e22e045c84f/users/6cebfb2bb96942c9b64720d9b8d9c6b2/roles/d67cdb8bae3648fd8f3589ab33176d14` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:14.669665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094ede5-a29e-4dbc-a943-d62cbb00c658 None admin] RBAC: Authorizing `identity:create_grant(project_id=59f00389bb404b4392d98e22e045c84f, user_id=6cebfb2bb96942c9b64720d9b8d9c6b2, role_id=d67cdb8bae3648fd8f3589ab33176d14)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:14.671125 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094ede5-a29e-4dbc-a943-d62cbb00c658 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:14.687678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 419/832] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:38:14 2022] PUT /identity/v3/projects/59f00389bb404b4392d98e22e045c84f/users/6cebfb2bb96942c9b64720d9b8d9c6b2/roles/d67cdb8bae3648fd8f3589ab33176d14 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:38:15.053024 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e38cc-98d8-4fe1-a730-7be2b57b5f80 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:15.054909 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e38cc-98d8-4fe1-a730-7be2b57b5f80 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:15.055191 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e38cc-98d8-4fe1-a730-7be2b57b5f80 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:15.056916 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 414/833] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:15.066188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947e0eb-85f1-45dd-8b8f-d7cc4d25332d None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:15.066188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947e0eb-85f1-45dd-8b8f-d7cc4d25332d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:15.066188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947e0eb-85f1-45dd-8b8f-d7cc4d25332d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:15.083064 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-e947e0eb-85f1-45dd-8b8f-d7cc4d25332d None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:15.084249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e947e0eb-85f1-45dd-8b8f-d7cc4d25332d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:15.158636 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 420/834] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:15.182835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e38cc-98d8-4fe1-a730-7be2b57b5f80 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:15.259946 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:15.301373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e38cc-98d8-4fe1-a730-7be2b57b5f80 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:15.337527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fee567-9fe4-4a16-94ae-a361c4b1f61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:15.338176 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fee567-9fe4-4a16-94ae-a361c4b1f61f None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:15.338461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fee567-9fe4-4a16-94ae-a361c4b1f61f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:15.338736 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fee567-9fe4-4a16-94ae-a361c4b1f61f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:15.349214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fee567-9fe4-4a16-94ae-a361c4b1f61f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-71fee567-9fe4-4a16-94ae-a361c4b1f61f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:15.354607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:15.355595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:15.356948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:15.356948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:15.356948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:15.356948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:15.356948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:15.356948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:15.356948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:15.356948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:15.356948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:15.356948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:15.356948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:15.356948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:15.356948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 415/835] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:38:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:15.363904 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947e0eb-85f1-45dd-8b8f-d7cc4d25332d None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:15.431067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:15.478310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947e0eb-85f1-45dd-8b8f-d7cc4d25332d None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:15.516758 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e8c64-f691-4b3a-a189-e0e68edcc701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:15.517306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e8c64-f691-4b3a-a189-e0e68edcc701 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:15.517585 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e8c64-f691-4b3a-a189-e0e68edcc701 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:15.517858 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e8c64-f691-4b3a-a189-e0e68edcc701 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:15.518582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e8c64-f691-4b3a-a189-e0e68edcc701 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:15.518884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e8c64-f691-4b3a-a189-e0e68edcc701 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:15.520260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e8c64-f691-4b3a-a189-e0e68edcc701 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:15.529903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 421/836] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:38:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:15.535549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fee567-9fe4-4a16-94ae-a361c4b1f61f None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:15.649776 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fee567-9fe4-4a16-94ae-a361c4b1f61f None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:15.680798 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:15.686063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd3ed5-7eae-4d47-b9f9-d7d85b8bf42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:15.686886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd3ed5-7eae-4d47-b9f9-d7d85b8bf42d None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:15.687427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd3ed5-7eae-4d47-b9f9-d7d85b8bf42d None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:15.687925 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd3ed5-7eae-4d47-b9f9-d7d85b8bf42d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:15.698662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd3ed5-7eae-4d47-b9f9-d7d85b8bf42d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:15.700322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd3ed5-7eae-4d47-b9f9-d7d85b8bf42d None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:15.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-aedd3ed5-7eae-4d47-b9f9-d7d85b8bf42d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:15.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:15.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:15.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:15.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:15.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:15.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:15.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:15.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:15.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:15.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:15.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:15.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:15.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:15.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:15.717859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:15.718975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:15.718975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:15.718975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:15.718975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:15.720848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 416/837] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:15.732415 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e8c64-f691-4b3a-a189-e0e68edcc701 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:15.848264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e8c64-f691-4b3a-a189-e0e68edcc701 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:15.884702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:15.892737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0f9af-b387-41db-8899-1230e27faf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:15.893698 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0f9af-b387-41db-8899-1230e27faf5b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:15.894049 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0f9af-b387-41db-8899-1230e27faf5b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:15.894317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0f9af-b387-41db-8899-1230e27faf5b None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:15.895020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0f9af-b387-41db-8899-1230e27faf5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:15.895319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0f9af-b387-41db-8899-1230e27faf5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:15.896788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0f9af-b387-41db-8899-1230e27faf5b None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:15.910948 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 422/838] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:15.917203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd3ed5-7eae-4d47-b9f9-d7d85b8bf42d None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:15.969345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea08518-b94a-4535-80d8-9b4ea560ac10 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:15.969676 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea08518-b94a-4535-80d8-9b4ea560ac10 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:15.969902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea08518-b94a-4535-80d8-9b4ea560ac10 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:15.971573 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 423/839] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:38:15.977043 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448b92d-f0f1-4986-a05f-acc85e3f6cd0 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:15.977327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448b92d-f0f1-4986-a05f-acc85e3f6cd0 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:15.977683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448b92d-f0f1-4986-a05f-acc85e3f6cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:15.996017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-c448b92d-f0f1-4986-a05f-acc85e3f6cd0 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:15.997250 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c448b92d-f0f1-4986-a05f-acc85e3f6cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:16.048247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd3ed5-7eae-4d47-b9f9-d7d85b8bf42d None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:16.082607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 424/840] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:16.100178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9152b-ea69-4394-baae-135b241e221b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:16.100178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9152b-ea69-4394-baae-135b241e221b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:16.100178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9152b-ea69-4394-baae-135b241e221b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:16.100178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9152b-ea69-4394-baae-135b241e221b None admin] PATH_INFO: `/v3/users/swift` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:16.107102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9152b-ea69-4394-baae-135b241e221b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:16.108765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9152b-ea69-4394-baae-135b241e221b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:16.109219 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448b92d-f0f1-4986-a05f-acc85e3f6cd0 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-1fa9152b-ea69-4394-baae-135b241e221b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:38:16.121718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:16.122759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:16.123791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:16.123791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:16.123791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:38:16.123791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:38:16.123791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:38:16.123791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:38:16.123791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:38:16.123791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:38:16.123791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 18:38:16.123791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:16.127505 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 417/841] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:38:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:16.129113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9152b-ea69-4394-baae-135b241e221b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:16.175186 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:16.229593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448b92d-f0f1-4986-a05f-acc85e3f6cd0 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:16.247917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:16.251390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9152b-ea69-4394-baae-135b241e221b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:16.268880 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c265ef-5fc6-452c-ba0b-827957c58561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:16.269883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c265ef-5fc6-452c-ba0b-827957c58561 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:16.270363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c265ef-5fc6-452c-ba0b-827957c58561 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:16.270834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c265ef-5fc6-452c-ba0b-827957c58561 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:16.271660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c265ef-5fc6-452c-ba0b-827957c58561 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:16.274591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c265ef-5fc6-452c-ba0b-827957c58561 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-53c265ef-5fc6-452c-ba0b-827957c58561 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:16.284152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:16.285298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:16.286132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:16.286132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:16.286132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:16.286132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:16.286132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:16.286132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:16.286132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:16.286132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:16.286132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:16.286132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:16.286132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:16.286132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:16.286132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 18:38:16.286132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:16.294042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 425/842] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:38:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:16.295402 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c265ef-5fc6-452c-ba0b-827957c58561 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:16.297041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45e873-c096-4837-b2b5-78baa6b53aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:16.297706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45e873-c096-4837-b2b5-78baa6b53aa0 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:16.297994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45e873-c096-4837-b2b5-78baa6b53aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:16.298274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45e873-c096-4837-b2b5-78baa6b53aa0 None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:16.299100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45e873-c096-4837-b2b5-78baa6b53aa0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:16.299412 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45e873-c096-4837-b2b5-78baa6b53aa0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:16.300891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45e873-c096-4837-b2b5-78baa6b53aa0 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:16.326207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 418/843] 10.222.0.69 () {56 vars in 1135 bytes} [Wed Aug 3 18:38:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:16.327643 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45e873-c096-4837-b2b5-78baa6b53aa0 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:16.411403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c265ef-5fc6-452c-ba0b-827957c58561 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:16.439488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:16.445972 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45e873-c096-4837-b2b5-78baa6b53aa0 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:16.453697 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce13ba-a706-4add-8bfd-54d050895673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:16.454999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce13ba-a706-4add-8bfd-54d050895673 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:16.455479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce13ba-a706-4add-8bfd-54d050895673 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:16.455946 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce13ba-a706-4add-8bfd-54d050895673 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:16.456847 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce13ba-a706-4add-8bfd-54d050895673 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:16.457347 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce13ba-a706-4add-8bfd-54d050895673 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:16.459086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce13ba-a706-4add-8bfd-54d050895673 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:16.470055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 426/844] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:38:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:16.471290 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce13ba-a706-4add-8bfd-54d050895673 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:16.486269 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b546e-0179-4fb9-9ef0-dd603a143332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:16.486896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b546e-0179-4fb9-9ef0-dd603a143332 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:16.487183 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b546e-0179-4fb9-9ef0-dd603a143332 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:16.487462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b546e-0179-4fb9-9ef0-dd603a143332 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:16.526795 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:16.568272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b546e-0179-4fb9-9ef0-dd603a143332 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:16.570623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b546e-0179-4fb9-9ef0-dd603a143332 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:16.588476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce13ba-a706-4add-8bfd-54d050895673 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:16.625078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63400525-3fe9-44b4-8b24-529e6c6972cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:16.626114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63400525-3fe9-44b4-8b24-529e6c6972cc None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:16.626593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63400525-3fe9-44b4-8b24-529e6c6972cc None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:16.627060 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63400525-3fe9-44b4-8b24-529e6c6972cc None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:16.628184 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63400525-3fe9-44b4-8b24-529e6c6972cc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:16.628738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63400525-3fe9-44b4-8b24-529e6c6972cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:16.630293 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63400525-3fe9-44b4-8b24-529e6c6972cc None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:16.641905 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 427/845] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:38:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:16.680341 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:16.691751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 419/846] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:38:16 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:38:16.693636 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63400525-3fe9-44b4-8b24-529e6c6972cc None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:16.704687 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072f634-1b1e-4296-87b4-f151b481834a None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:16.705002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072f634-1b1e-4296-87b4-f151b481834a None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:16.705248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072f634-1b1e-4296-87b4-f151b481834a None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:16.706983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 420/847] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:38:16.714746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0c0cb-9dba-421e-b53d-e6235659388a None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:16.715078 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0c0cb-9dba-421e-b53d-e6235659388a None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:16.715310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0c0cb-9dba-421e-b53d-e6235659388a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:16.734633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-97b0c0cb-9dba-421e-b53d-e6235659388a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:16.735892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-97b0c0cb-9dba-421e-b53d-e6235659388a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:16.821966 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 421/848] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:16.833376 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63400525-3fe9-44b4-8b24-529e6c6972cc None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:16.844434 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0c0cb-9dba-421e-b53d-e6235659388a None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:16.845913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:16.884516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187e87a-672d-48d2-8e27-e521926f263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:16.885187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187e87a-672d-48d2-8e27-e521926f263b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:16.885468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187e87a-672d-48d2-8e27-e521926f263b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:16.886005 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187e87a-672d-48d2-8e27-e521926f263b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:16.899241 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187e87a-672d-48d2-8e27-e521926f263b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:16.901105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187e87a-672d-48d2-8e27-e521926f263b None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:16.916983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-3187e87a-672d-48d2-8e27-e521926f263b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:16.916983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:16.916983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:16.916983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:16.916983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:16.916983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:16.916983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:16.916983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:16.916983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:16.916983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:16.916983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:16.916983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:16.916983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:16.916983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:16.916983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:16.918082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:16.919088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:16.919088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:16.919088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:16.919088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:16.919088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 428/849] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:16.920629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d08e7-ec6f-430c-8795-28495494f41a None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:16.921054 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d08e7-ec6f-430c-8795-28495494f41a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:16.921466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d08e7-ec6f-430c-8795-28495494f41a None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:16.923644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 429/850] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:16.926416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d08e7-ec6f-430c-8795-28495494f41a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:16.982022 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0c0cb-9dba-421e-b53d-e6235659388a None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:17.012100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:17.020968 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47a75b-7fbd-426a-95e9-95139f813f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:17.021409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47a75b-7fbd-426a-95e9-95139f813f47 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:17.021693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47a75b-7fbd-426a-95e9-95139f813f47 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:17.021982 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47a75b-7fbd-426a-95e9-95139f813f47 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:17.023189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47a75b-7fbd-426a-95e9-95139f813f47 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:17.024766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47a75b-7fbd-426a-95e9-95139f813f47 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-3e47a75b-7fbd-426a-95e9-95139f813f47 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:17.034451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:17.035465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:17.036335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 422/851] 10.222.0.69 () {56 vars in 1132 bytes} [Wed Aug 3 18:38:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:17.037248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e931714-296c-4c8c-a759-0415e6641db8 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:17.037248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e931714-296c-4c8c-a759-0415e6641db8 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:17.037382 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e931714-296c-4c8c-a759-0415e6641db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:17.054690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d08e7-ec6f-430c-8795-28495494f41a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:17.057047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-3e931714-296c-4c8c-a759-0415e6641db8 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:17.058240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-3e931714-296c-4c8c-a759-0415e6641db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:17.094117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf67fd-38f1-4e73-b9db-6fbbf5988d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:17.095109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf67fd-38f1-4e73-b9db-6fbbf5988d3e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:17.095109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf67fd-38f1-4e73-b9db-6fbbf5988d3e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:17.095334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf67fd-38f1-4e73-b9db-6fbbf5988d3e None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:17.095963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf67fd-38f1-4e73-b9db-6fbbf5988d3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:17.096353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf67fd-38f1-4e73-b9db-6fbbf5988d3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:17.097743 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf67fd-38f1-4e73-b9db-6fbbf5988d3e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:17.114482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 430/852] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:17.114482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf67fd-38f1-4e73-b9db-6fbbf5988d3e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:17.134719 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:17.136274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 423/853] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:17.150714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e931714-296c-4c8c-a759-0415e6641db8 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:17.224863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf67fd-38f1-4e73-b9db-6fbbf5988d3e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:17.268759 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8053e-10a5-4a27-825e-c12f1bc7c2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:17.269145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e931714-296c-4c8c-a759-0415e6641db8 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:17.269508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8053e-10a5-4a27-825e-c12f1bc7c2d7 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:17.269757 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8053e-10a5-4a27-825e-c12f1bc7c2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:17.269995 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8053e-10a5-4a27-825e-c12f1bc7c2d7 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:17.270686 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8053e-10a5-4a27-825e-c12f1bc7c2d7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:17.270996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8053e-10a5-4a27-825e-c12f1bc7c2d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:17.272501 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8053e-10a5-4a27-825e-c12f1bc7c2d7 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:17.280338 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 431/854] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Aug 3 18:38:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:17.280338 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8053e-10a5-4a27-825e-c12f1bc7c2d7 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:17.300999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:17.315136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc57347-4a0d-4737-9936-7d6273ba575e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:17.315780 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc57347-4a0d-4737-9936-7d6273ba575e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:17.316073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc57347-4a0d-4737-9936-7d6273ba575e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:17.316361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc57347-4a0d-4737-9936-7d6273ba575e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:17.325114 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc57347-4a0d-4737-9936-7d6273ba575e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:17.326519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc57347-4a0d-4737-9936-7d6273ba575e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:17.338856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-abc57347-4a0d-4737-9936-7d6273ba575e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:17.338856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:17.338856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:17.338856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:17.338856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:17.338856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:17.338856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:17.338856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:17.338856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:17.338856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:17.338856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:17.338856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:17.338856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:17.338856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:17.338856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:17.339783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:17.341128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:17.341128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:17.341128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:17.341128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:17.341128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:17.341128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:17.341128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:17.341128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:17.341128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 424/855] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:17.341128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc57347-4a0d-4737-9936-7d6273ba575e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:17.377262 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:17.394743 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8053e-10a5-4a27-825e-c12f1bc7c2d7 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:17.438345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402ad88-135b-42f9-b6de-a9aca50dcfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:17.439255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402ad88-135b-42f9-b6de-a9aca50dcfae None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:17.439546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402ad88-135b-42f9-b6de-a9aca50dcfae None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:17.440249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402ad88-135b-42f9-b6de-a9aca50dcfae None admin] PATH_INFO: `/v3/roles/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:17.448843 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc57347-4a0d-4737-9936-7d6273ba575e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:17.453921 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402ad88-135b-42f9-b6de-a9aca50dcfae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-5402ad88-135b-42f9-b6de-a9aca50dcfae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:17.459354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:17.460958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:17.462591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:17.462591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:17.462591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:17.462591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:17.462591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:17.462591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:17.462591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:17.462591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:17.462591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:17.462591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:17.462591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:38:17.462591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:17.462591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 432/856] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:38:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:17.462591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402ad88-135b-42f9-b6de-a9aca50dcfae None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:17.491787 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fded7-ad26-497b-bf96-891e075139b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:17.492377 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fded7-ad26-497b-bf96-891e075139b3 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:17.492671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fded7-ad26-497b-bf96-891e075139b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:17.492958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fded7-ad26-497b-bf96-891e075139b3 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:17.493639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5fded7-ad26-497b-bf96-891e075139b3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:17.493945 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5fded7-ad26-497b-bf96-891e075139b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:17.495658 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5fded7-ad26-497b-bf96-891e075139b3 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:17.500349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 425/857] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Aug 3 18:38:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:17.501127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fded7-ad26-497b-bf96-891e075139b3 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:17.566469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:17.605936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402ad88-135b-42f9-b6de-a9aca50dcfae None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:17.646785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059a074-eeb6-4eb5-bfc6-c0156b0cb190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:17.648212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059a074-eeb6-4eb5-bfc6-c0156b0cb190 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:17.648548 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059a074-eeb6-4eb5-bfc6-c0156b0cb190 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:17.648829 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059a074-eeb6-4eb5-bfc6-c0156b0cb190 None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:17.649695 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059a074-eeb6-4eb5-bfc6-c0156b0cb190 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:17.649913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059a074-eeb6-4eb5-bfc6-c0156b0cb190 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:17.651391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059a074-eeb6-4eb5-bfc6-c0156b0cb190 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:17.653242 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fded7-ad26-497b-bf96-891e075139b3 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:17.666133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 433/858] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:38:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:17.672302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059a074-eeb6-4eb5-bfc6-c0156b0cb190 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:17.676095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:17.705600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec7da5-197f-4bbc-bc6a-1d2188f62a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:17.706438 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec7da5-197f-4bbc-bc6a-1d2188f62a93 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:17.706719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec7da5-197f-4bbc-bc6a-1d2188f62a93 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:17.706991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec7da5-197f-4bbc-bc6a-1d2188f62a93 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:17.707720 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec7da5-197f-4bbc-bc6a-1d2188f62a93 None admin] RBAC: Adding query filter params (name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:17.708025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec7da5-197f-4bbc-bc6a-1d2188f62a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:17.709567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec7da5-197f-4bbc-bc6a-1d2188f62a93 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:17.719604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 426/859] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:38:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:17.727549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec7da5-197f-4bbc-bc6a-1d2188f62a93 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:17.799655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059a074-eeb6-4eb5-bfc6-c0156b0cb190 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:17.839532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ddda5-412b-410e-bb59-2075450c292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:17.840181 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ddda5-412b-410e-bb59-2075450c292d None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:17.840499 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ddda5-412b-410e-bb59-2075450c292d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:17.840779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ddda5-412b-410e-bb59-2075450c292d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:17.841611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ddda5-412b-410e-bb59-2075450c292d None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=96ed059287274ca392c6abb893551094) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:17.841931 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ddda5-412b-410e-bb59-2075450c292d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:17.843640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ddda5-412b-410e-bb59-2075450c292d None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:17.851459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 434/860] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:38:17 2022] GET /identity/v3/role_assignments?user.id=96ed059287274ca392c6abb893551094&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=19557e941d6f475ba92cf862846f9872 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:17.859659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec7da5-197f-4bbc-bc6a-1d2188f62a93 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:17.872101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:17.908178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a17ef-5fef-43ba-b648-3caf15947b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:17.908889 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a17ef-5fef-43ba-b648-3caf15947b6e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:17.909180 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a17ef-5fef-43ba-b648-3caf15947b6e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:17.909461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a17ef-5fef-43ba-b648-3caf15947b6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:17.922599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a17ef-5fef-43ba-b648-3caf15947b6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:17.924082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a17ef-5fef-43ba-b648-3caf15947b6e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:17.942083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-608a17ef-5fef-43ba-b648-3caf15947b6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:17.942083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:17.942083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:17.942083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:17.942083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:17.942083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:17.942083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:17.942083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:17.942083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:17.942083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:17.942083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:17.942083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:17.942083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:17.942083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:17.942083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:17.943062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:17.943969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:17.943969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:17.943969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:17.943969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:17.943969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 427/861] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:17.952517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ddda5-412b-410e-bb59-2075450c292d None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:18.031951 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:18.070679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ddda5-412b-410e-bb59-2075450c292d None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:18.106292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4e058-4193-423e-a5ae-a15f82c32092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:18.107013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4e058-4193-423e-a5ae-a15f82c32092 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:18.107364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4e058-4193-423e-a5ae-a15f82c32092 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:18.107707 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4e058-4193-423e-a5ae-a15f82c32092 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:18.108506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4e058-4193-423e-a5ae-a15f82c32092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:18.108879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4e058-4193-423e-a5ae-a15f82c32092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:18.110317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4e058-4193-423e-a5ae-a15f82c32092 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:18.124032 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 435/862] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:18.129950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a17ef-5fef-43ba-b648-3caf15947b6e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:18.236731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a17ef-5fef-43ba-b648-3caf15947b6e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:18.277370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c625dc-f398-4c7d-9e4a-f10f498f8048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:18.278025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c625dc-f398-4c7d-9e4a-f10f498f8048 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:18.278310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c625dc-f398-4c7d-9e4a-f10f498f8048 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:18.278590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c625dc-f398-4c7d-9e4a-f10f498f8048 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:18.287023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c625dc-f398-4c7d-9e4a-f10f498f8048 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:18.289101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c625dc-f398-4c7d-9e4a-f10f498f8048 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-71c625dc-f398-4c7d-9e4a-f10f498f8048 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:38:18.301074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:18.303258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:18.306610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:18.306610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:18.306610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:38:18.306610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:38:18.306610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:38:18.306610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:38:18.306610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:38:18.306610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:38:18.306610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 18:38:18.306610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:18.306610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 428/863] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Aug 3 18:38:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:18.312526 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4e058-4193-423e-a5ae-a15f82c32092 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:18.421197 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4e058-4193-423e-a5ae-a15f82c32092 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:18.449985 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:18.458280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e60a9f-014e-4212-8bdf-6f9e91d7cc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:18.458929 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e60a9f-014e-4212-8bdf-6f9e91d7cc23 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:18.459188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e60a9f-014e-4212-8bdf-6f9e91d7cc23 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:18.459466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e60a9f-014e-4212-8bdf-6f9e91d7cc23 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:18.460379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e60a9f-014e-4212-8bdf-6f9e91d7cc23 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:18.460693 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e60a9f-014e-4212-8bdf-6f9e91d7cc23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:18.462222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e60a9f-014e-4212-8bdf-6f9e91d7cc23 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:18.483462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 436/864] 10.222.0.69 () {56 vars in 1149 bytes} [Wed Aug 3 18:38:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:18.489375 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c625dc-f398-4c7d-9e4a-f10f498f8048 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:18.516028 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:18.606799 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c625dc-f398-4c7d-9e4a-f10f498f8048 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:18.643166 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7993265-7541-41ac-a973-4ec53be0842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:18.643795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7993265-7541-41ac-a973-4ec53be0842d None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:18.644082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7993265-7541-41ac-a973-4ec53be0842d None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:18.644388 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7993265-7541-41ac-a973-4ec53be0842d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:18.717929 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7993265-7541-41ac-a973-4ec53be0842d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:18.719398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7993265-7541-41ac-a973-4ec53be0842d None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:18.742899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:18.784517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536326e4-354f-43c8-b788-4b1d1552f478 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:18.784864 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536326e4-354f-43c8-b788-4b1d1552f478 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:18.785087 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536326e4-354f-43c8-b788-4b1d1552f478 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:18.786693 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 437/865] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:18.792751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428855ff-882b-46bb-aaac-f62226c886a3 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:18.793035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428855ff-882b-46bb-aaac-f62226c886a3 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:18.793310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428855ff-882b-46bb-aaac-f62226c886a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:18.811058 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-428855ff-882b-46bb-aaac-f62226c886a3 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:18.812255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-428855ff-882b-46bb-aaac-f62226c886a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:18.838510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 429/866] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:38:18 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:38:18.843602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7993265-7541-41ac-a973-4ec53be0842d None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:18.886044 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 438/867] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:18.909142 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428855ff-882b-46bb-aaac-f62226c886a3 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:18.971842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7993265-7541-41ac-a973-4ec53be0842d None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:19.006342 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:19.015802 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c5760-0e97-4d81-9d3a-1598d698009d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:19.017161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c5760-0e97-4d81-9d3a-1598d698009d None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:19.017454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c5760-0e97-4d81-9d3a-1598d698009d None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:19.017739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c5760-0e97-4d81-9d3a-1598d698009d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:19.029507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c5760-0e97-4d81-9d3a-1598d698009d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:19.031231 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c5760-0e97-4d81-9d3a-1598d698009d None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:19.045462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428855ff-882b-46bb-aaac-f62226c886a3 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:19.048934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-017c5760-0e97-4d81-9d3a-1598d698009d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:19.048934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:19.048934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:19.048934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:19.048934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:19.048934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:19.048934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:19.048934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:19.048934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:19.048934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:19.048934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:19.048934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:19.048934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:19.048934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:19.048934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:19.049813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:19.050710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:19.050710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:19.050710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:19.050710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:19.053019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 430/868] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:19.062962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c5760-0e97-4d81-9d3a-1598d698009d None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:19.066569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:19.095032 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ddbad-18ef-4b97-a5d6-871ed73c3711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:19.095938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ddbad-18ef-4b97-a5d6-871ed73c3711 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:19.096258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ddbad-18ef-4b97-a5d6-871ed73c3711 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:19.096656 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ddbad-18ef-4b97-a5d6-871ed73c3711 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:19.097467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ddbad-18ef-4b97-a5d6-871ed73c3711 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:19.099034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ddbad-18ef-4b97-a5d6-871ed73c3711 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:19.118241 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 439/869] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Aug 3 18:38:18 2022] POST /identity/v3/services => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:38:19.193565 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c5760-0e97-4d81-9d3a-1598d698009d None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:19.241485 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d7da7-f6f7-4930-a740-217884a74579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:19.242206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d7da7-f6f7-4930-a740-217884a74579 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:19.242291 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d7da7-f6f7-4930-a740-217884a74579 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:19.242527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d7da7-f6f7-4930-a740-217884a74579 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:19.243331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d7da7-f6f7-4930-a740-217884a74579 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:19.243647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d7da7-f6f7-4930-a740-217884a74579 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:19.245196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d7da7-f6f7-4930-a740-217884a74579 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:19.261297 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 431/870] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:19.269495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ddbad-18ef-4b97-a5d6-871ed73c3711 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:19.386620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ddbad-18ef-4b97-a5d6-871ed73c3711 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:19.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e29d0f-af48-4445-af73-f2be78e3793e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:19.424036 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e29d0f-af48-4445-af73-f2be78e3793e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:19.424408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e29d0f-af48-4445-af73-f2be78e3793e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:19.424688 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e29d0f-af48-4445-af73-f2be78e3793e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:19.432841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e29d0f-af48-4445-af73-f2be78e3793e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:19.434378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e29d0f-af48-4445-af73-f2be78e3793e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:19.447496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-b4e29d0f-af48-4445-af73-f2be78e3793e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:19.447496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:19.447496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:19.447496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:19.447496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:19.447496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:19.447496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:19.447496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:19.447496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:19.447496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:19.447496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:19.447496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:19.447496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:19.447496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:19.447496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:19.448456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:19.449369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:19.449369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:19.449369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:19.449369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:19.449369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:19.449369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:19.449369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:19.449369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:19.449369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 440/871] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:19.454791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d7da7-f6f7-4930-a740-217884a74579 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:19.484724 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:19.565702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d7da7-f6f7-4930-a740-217884a74579 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:19.603534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e3263-aaae-4822-abfb-84aa58327059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:19.604168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e3263-aaae-4822-abfb-84aa58327059 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:19.604483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e3263-aaae-4822-abfb-84aa58327059 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:19.604759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e3263-aaae-4822-abfb-84aa58327059 None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:19.605531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e3263-aaae-4822-abfb-84aa58327059 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:19.605839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e3263-aaae-4822-abfb-84aa58327059 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:19.607275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e3263-aaae-4822-abfb-84aa58327059 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:19.621188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 432/872] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:38:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:19.626828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e29d0f-af48-4445-af73-f2be78e3793e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:19.668680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:19.744725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e29d0f-af48-4445-af73-f2be78e3793e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:19.755714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70bebe-5d19-4090-8b92-72b190b2b5c1 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:19.756353 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70bebe-5d19-4090-8b92-72b190b2b5c1 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:19.757399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70bebe-5d19-4090-8b92-72b190b2b5c1 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:19.759252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 433/873] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:19.764752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2945755-c262-4c8f-8460-cd7281a4327e None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:19.765338 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2945755-c262-4c8f-8460-cd7281a4327e None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:19.765824 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2945755-c262-4c8f-8460-cd7281a4327e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:19.782334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236a458-c898-4106-ba82-9c1341a907d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:19.782968 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236a458-c898-4106-ba82-9c1341a907d0 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:19.783251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236a458-c898-4106-ba82-9c1341a907d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:19.783525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236a458-c898-4106-ba82-9c1341a907d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:19.784342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236a458-c898-4106-ba82-9c1341a907d0 None admin] RBAC: Adding query filter params (role.id=d67cdb8bae3648fd8f3589ab33176d14, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=6cebfb2bb96942c9b64720d9b8d9c6b2) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:19.784656 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236a458-c898-4106-ba82-9c1341a907d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:19.786095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236a458-c898-4106-ba82-9c1341a907d0 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:19.786504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-e2945755-c262-4c8f-8460-cd7281a4327e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:19.788609 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e2945755-c262-4c8f-8460-cd7281a4327e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:19.793501 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 441/874] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:38:19 2022] GET /identity/v3/role_assignments?user.id=6cebfb2bb96942c9b64720d9b8d9c6b2&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=d67cdb8bae3648fd8f3589ab33176d14 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:19.866673 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 434/875] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:19 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:19.891606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236a458-c898-4106-ba82-9c1341a907d0 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:20.004828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:20.008847 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236a458-c898-4106-ba82-9c1341a907d0 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:20.045353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ad1a3-be7d-4b29-9811-858da2c13cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:20.045988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ad1a3-be7d-4b29-9811-858da2c13cf8 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:20.046273 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ad1a3-be7d-4b29-9811-858da2c13cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:20.046555 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ad1a3-be7d-4b29-9811-858da2c13cf8 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:20.047302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ad1a3-be7d-4b29-9811-858da2c13cf8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:20.048773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ad1a3-be7d-4b29-9811-858da2c13cf8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:20.066442 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 442/876] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Aug 3 18:38:19 2022] POST /identity/v3/services => generated 269 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:38:20.115824 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73407a0a-d503-46bb-a397-5ee262711a94 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:20.116445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73407a0a-d503-46bb-a397-5ee262711a94 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:20.116445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73407a0a-d503-46bb-a397-5ee262711a94 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:20.118029 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 435/877] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:20.124263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42809bea-087d-4a2b-b3f2-f054c9a04464 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:20.124866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42809bea-087d-4a2b-b3f2-f054c9a04464 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:20.125320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42809bea-087d-4a2b-b3f2-f054c9a04464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:20.146549 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-42809bea-087d-4a2b-b3f2-f054c9a04464 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:20.149599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-42809bea-087d-4a2b-b3f2-f054c9a04464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:20.237345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 443/878] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:20.260998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73407a0a-d503-46bb-a397-5ee262711a94 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:20.281107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:20.383276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73407a0a-d503-46bb-a397-5ee262711a94 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:20.422870 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d0605-3edd-4708-ba06-9a9e44f8c7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:20.423928 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d0605-3edd-4708-ba06-9a9e44f8c7b8 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:20.424421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d0605-3edd-4708-ba06-9a9e44f8c7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:20.424876 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d0605-3edd-4708-ba06-9a9e44f8c7b8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:20.425698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d0605-3edd-4708-ba06-9a9e44f8c7b8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:20.427380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d0605-3edd-4708-ba06-9a9e44f8c7b8 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-007d0605-3edd-4708-ba06-9a9e44f8c7b8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:20.438427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:20.439458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:20.440339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:20.440339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:20.440339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:20.440339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:20.440339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:20.440339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:20.440339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:20.440339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:20.440339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:20.440339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:20.440339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:20.440339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:20.440339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 18:38:20.440339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:20.442961 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 436/879] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Aug 3 18:38:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:20.449699 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42809bea-087d-4a2b-b3f2-f054c9a04464 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:20.542912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:20.560060 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42809bea-087d-4a2b-b3f2-f054c9a04464 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:20.597451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7d471-5c73-4e53-87ce-c0630c09ba78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:20.598375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7d471-5c73-4e53-87ce-c0630c09ba78 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:20.598837 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7d471-5c73-4e53-87ce-c0630c09ba78 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:20.599276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7d471-5c73-4e53-87ce-c0630c09ba78 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:20.600670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7d471-5c73-4e53-87ce-c0630c09ba78 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:20.601324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7d471-5c73-4e53-87ce-c0630c09ba78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:20.602918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7d471-5c73-4e53-87ce-c0630c09ba78 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:20.608102 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 444/880] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Aug 3 18:38:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:20.613946 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d0605-3edd-4708-ba06-9a9e44f8c7b8 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:20.707893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:20.733489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d0605-3edd-4708-ba06-9a9e44f8c7b8 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:20.775735 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bef957-096f-43d2-9932-a89848051e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:20.776708 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bef957-096f-43d2-9932-a89848051e3b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:20.777224 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bef957-096f-43d2-9932-a89848051e3b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:20.777648 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bef957-096f-43d2-9932-a89848051e3b None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:20.778502 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bef957-096f-43d2-9932-a89848051e3b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:20.778959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bef957-096f-43d2-9932-a89848051e3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:20.780636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bef957-096f-43d2-9932-a89848051e3b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:20.786610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 437/881] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Aug 3 18:38:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:21.335053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a5792-3662-411e-a263-404d2bb8cedd None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:21.335742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a5792-3662-411e-a263-404d2bb8cedd None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:21.335742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a5792-3662-411e-a263-404d2bb8cedd None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:21.337379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 445/882] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:21.342568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba699cb9-8249-41ae-adf4-7f8d000b6904 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:21.342980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba699cb9-8249-41ae-adf4-7f8d000b6904 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:21.343121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba699cb9-8249-41ae-adf4-7f8d000b6904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:21.374937 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ba699cb9-8249-41ae-adf4-7f8d000b6904 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:21.376132 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ba699cb9-8249-41ae-adf4-7f8d000b6904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:21.473188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 438/883] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:21.497481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a5792-3662-411e-a263-404d2bb8cedd None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:21.608437 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a5792-3662-411e-a263-404d2bb8cedd None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:21.645176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f213741-c79c-48fb-a8fe-89dd0c7be4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:21.645831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f213741-c79c-48fb-a8fe-89dd0c7be4af None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:21.646115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f213741-c79c-48fb-a8fe-89dd0c7be4af None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:21.646393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f213741-c79c-48fb-a8fe-89dd0c7be4af None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:21.647622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f213741-c79c-48fb-a8fe-89dd0c7be4af None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:21.650010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f213741-c79c-48fb-a8fe-89dd0c7be4af None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-0f213741-c79c-48fb-a8fe-89dd0c7be4af None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:21.659687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:21.660729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:21.661587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:21.661587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:21.661587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:21.661587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:21.661587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:21.661587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:21.661587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:21.661587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:21.661587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:21.661587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:21.661587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:21.661587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:21.661587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 18:38:21.661587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:21.661587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 446/884] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:38:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:21.667100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba699cb9-8249-41ae-adf4-7f8d000b6904 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:21.762294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:21.787768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba699cb9-8249-41ae-adf4-7f8d000b6904 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:21.823404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e4829-2bee-449a-aa54-e4b7138c79dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:21.824034 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e4829-2bee-449a-aa54-e4b7138c79dc None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:21.824333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e4829-2bee-449a-aa54-e4b7138c79dc None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:21.824613 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e4829-2bee-449a-aa54-e4b7138c79dc None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:21.825301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e4829-2bee-449a-aa54-e4b7138c79dc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:21.825605 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e4829-2bee-449a-aa54-e4b7138c79dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:21.827029 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e4829-2bee-449a-aa54-e4b7138c79dc None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:21.832264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 439/885] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:38:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:21.838494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f213741-c79c-48fb-a8fe-89dd0c7be4af None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:21.858898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:21.968040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f213741-c79c-48fb-a8fe-89dd0c7be4af None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:22.017583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee62bd8-e161-4e27-9a36-c0e11fb667f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:22.018214 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee62bd8-e161-4e27-9a36-c0e11fb667f2 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:22.018498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee62bd8-e161-4e27-9a36-c0e11fb667f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:22.018773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee62bd8-e161-4e27-9a36-c0e11fb667f2 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:22.019467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee62bd8-e161-4e27-9a36-c0e11fb667f2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:22.019774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee62bd8-e161-4e27-9a36-c0e11fb667f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:22.021186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee62bd8-e161-4e27-9a36-c0e11fb667f2 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:22.027158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 447/886] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:38:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:22.032872 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e4829-2bee-449a-aa54-e4b7138c79dc None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:22.125537 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a880e-01a6-4647-968b-80743f543088 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:22.125829 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a880e-01a6-4647-968b-80743f543088 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:22.126025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a880e-01a6-4647-968b-80743f543088 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:22.127507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 448/887] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:22.133165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee5a13-bb18-4d0b-80b1-4065f7194f0e None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:22.133453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee5a13-bb18-4d0b-80b1-4065f7194f0e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:22.133734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee5a13-bb18-4d0b-80b1-4065f7194f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:22.141633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e4829-2bee-449a-aa54-e4b7138c79dc None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:22.152026 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-2eee5a13-bb18-4d0b-80b1-4065f7194f0e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:22.153286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2eee5a13-bb18-4d0b-80b1-4065f7194f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:22.176417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9eae02-015d-42d6-8127-de25ac3a5a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:22.177046 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9eae02-015d-42d6-8127-de25ac3a5a2e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:22.177329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9eae02-015d-42d6-8127-de25ac3a5a2e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:22.177603 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9eae02-015d-42d6-8127-de25ac3a5a2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:22.178436 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9eae02-015d-42d6-8127-de25ac3a5a2e None admin] RBAC: Adding query filter params (interface=public, service_id=91106d09913e49f0836f8852c6285b66, region_id=RegionOne) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:22.178746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9eae02-015d-42d6-8127-de25ac3a5a2e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:22.180320 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9eae02-015d-42d6-8127-de25ac3a5a2e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:22.186088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 440/888] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Aug 3 18:38:22 2022] GET /identity/v3/endpoints?service_id=91106d09913e49f0836f8852c6285b66&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:22.191393 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9eae02-015d-42d6-8127-de25ac3a5a2e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:22.216641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:22.236104 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 449/889] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:22.254211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee5a13-bb18-4d0b-80b1-4065f7194f0e None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:22.265989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:22.305225 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9eae02-015d-42d6-8127-de25ac3a5a2e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:22.340888 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e10c7-7da5-46d0-bedc-0f01acdec588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:22.341514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e10c7-7da5-46d0-bedc-0f01acdec588 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:22.341800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e10c7-7da5-46d0-bedc-0f01acdec588 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:22.342072 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e10c7-7da5-46d0-bedc-0f01acdec588 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:22.342730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e10c7-7da5-46d0-bedc-0f01acdec588 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:22.344091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e10c7-7da5-46d0-bedc-0f01acdec588 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:22.350550 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 441/890] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Aug 3 18:38:22 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:38:22.351565 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a0681-8315-40ac-ba71-af4eb09753a6 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:22.351853 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a0681-8315-40ac-ba71-af4eb09753a6 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:22.352168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a0681-8315-40ac-ba71-af4eb09753a6 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:22.353766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 442/891] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:22.362227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a526a00-5126-42cc-a655-c06372f662e4 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:22.362784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a526a00-5126-42cc-a655-c06372f662e4 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:22.363252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a526a00-5126-42cc-a655-c06372f662e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:22.376335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee5a13-bb18-4d0b-80b1-4065f7194f0e None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:22.385705 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-0a526a00-5126-42cc-a655-c06372f662e4 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:22.387482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-0a526a00-5126-42cc-a655-c06372f662e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:22.416850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101acd5-4477-4f50-b251-d499b165765f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:22.417531 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101acd5-4477-4f50-b251-d499b165765f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:22.417883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101acd5-4477-4f50-b251-d499b165765f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:22.418165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101acd5-4477-4f50-b251-d499b165765f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:22.431307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101acd5-4477-4f50-b251-d499b165765f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-0101acd5-4477-4f50-b251-d499b165765f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:22.436734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:22.437762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:22.438632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:22.438632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:22.438632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:22.438632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:22.438632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:22.438632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:22.438632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:22.438632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:22.438632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:22.438632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:22.438632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 18:38:22.438632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:22.443145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 450/892] 10.222.0.69 () {56 vars in 1120 bytes} [Wed Aug 3 18:38:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:22.444331 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101acd5-4477-4f50-b251-d499b165765f None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:22.480744 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:22.495027 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 443/893] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:22.517893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a526a00-5126-42cc-a655-c06372f662e4 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:22.537418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:22.584972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101acd5-4477-4f50-b251-d499b165765f None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:22.630273 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d0b69-e1d5-4b61-927e-cf59e5005734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:22.631114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d0b69-e1d5-4b61-927e-cf59e5005734 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:22.631693 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d0b69-e1d5-4b61-927e-cf59e5005734 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:22.632062 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d0b69-e1d5-4b61-927e-cf59e5005734 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:22.632938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d0b69-e1d5-4b61-927e-cf59e5005734 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:22.633330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d0b69-e1d5-4b61-927e-cf59e5005734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:22.635304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d0b69-e1d5-4b61-927e-cf59e5005734 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:22.644171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a526a00-5126-42cc-a655-c06372f662e4 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:22.649984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 451/894] 10.222.0.69 () {56 vars in 1115 bytes} [Wed Aug 3 18:38:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:22.651015 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d0b69-e1d5-4b61-927e-cf59e5005734 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:22.683423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57398667-6c9f-4910-b060-829eea16c816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:22.684694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57398667-6c9f-4910-b060-829eea16c816 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:22.685167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57398667-6c9f-4910-b060-829eea16c816 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:22.685775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57398667-6c9f-4910-b060-829eea16c816 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:22.686907 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57398667-6c9f-4910-b060-829eea16c816 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:22.688757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57398667-6c9f-4910-b060-829eea16c816 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-57398667-6c9f-4910-b060-829eea16c816 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:22.701167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:22.702188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:22.703074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:22.703074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:22.703074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:22.703074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:22.703074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:22.703074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:22.703074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:22.703074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:22.703074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:22.703074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:22.703074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:22.703074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:22.703074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:22.703074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 18:38:22.703074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:22.706234 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 444/895] 10.222.0.69 () {56 vars in 1132 bytes} [Wed Aug 3 18:38:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:22.712787 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57398667-6c9f-4910-b060-829eea16c816 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:22.779658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d0b69-e1d5-4b61-927e-cf59e5005734 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:22.782457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:22.816067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:22.823832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ba58e-adec-472f-8827-723848798773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:22.825093 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ba58e-adec-472f-8827-723848798773 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:22.825535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ba58e-adec-472f-8827-723848798773 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:22.825963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ba58e-adec-472f-8827-723848798773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:22.840748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ba58e-adec-472f-8827-723848798773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:22.842984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ba58e-adec-472f-8827-723848798773 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:22.844012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57398667-6c9f-4910-b060-829eea16c816 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:22.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-5a8ba58e-adec-472f-8827-723848798773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:22.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:22.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:22.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:22.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:22.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:22.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:22.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:22.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:22.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:22.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:22.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:22.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:22.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:22.861794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:22.862903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:22.864012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:22.864012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:22.864012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:22.864012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:22.866679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 452/896] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:22.875080 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ba58e-adec-472f-8827-723848798773 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:22.895282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93194d59-f419-4214-97d9-4e6d14626761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:22.896105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93194d59-f419-4214-97d9-4e6d14626761 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:22.896433 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93194d59-f419-4214-97d9-4e6d14626761 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:22.896686 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93194d59-f419-4214-97d9-4e6d14626761 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:22.897853 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93194d59-f419-4214-97d9-4e6d14626761 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:22.898137 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93194d59-f419-4214-97d9-4e6d14626761 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:22.900413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93194d59-f419-4214-97d9-4e6d14626761 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:22.906881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 445/897] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Aug 3 18:38:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:22.910310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe10b6-b303-42e8-9849-78b5e2798ed5 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:22.910310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe10b6-b303-42e8-9849-78b5e2798ed5 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:22.910310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe10b6-b303-42e8-9849-78b5e2798ed5 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:22.910744 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 446/898] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:22.918942 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe10b6-b303-42e8-9849-78b5e2798ed5 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:22.987999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ba58e-adec-472f-8827-723848798773 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:23.022916 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61acd4-ada4-4ff9-94a5-faa6d52cfdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:23.023541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61acd4-ada4-4ff9-94a5-faa6d52cfdd8 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:23.023872 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61acd4-ada4-4ff9-94a5-faa6d52cfdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:23.024181 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61acd4-ada4-4ff9-94a5-faa6d52cfdd8 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:23.024906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61acd4-ada4-4ff9-94a5-faa6d52cfdd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:23.025210 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61acd4-ada4-4ff9-94a5-faa6d52cfdd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:23.026569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61acd4-ada4-4ff9-94a5-faa6d52cfdd8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:23.034324 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe10b6-b303-42e8-9849-78b5e2798ed5 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:23.039508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 453/899] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:23.040882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb623e77-2092-402f-ace5-9bcd65c4d28d None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:23.041177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb623e77-2092-402f-ace5-9bcd65c4d28d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:23.041448 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb623e77-2092-402f-ace5-9bcd65c4d28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:23.061227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-bb623e77-2092-402f-ace5-9bcd65c4d28d None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:23.062411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-bb623e77-2092-402f-ace5-9bcd65c4d28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:23.068957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:23.080751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e04f7-664c-4883-b3ed-96f66cf9e7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:23.081482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e04f7-664c-4883-b3ed-96f66cf9e7e8 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:23.081775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e04f7-664c-4883-b3ed-96f66cf9e7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:23.082056 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e04f7-664c-4883-b3ed-96f66cf9e7e8 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:23.082742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e04f7-664c-4883-b3ed-96f66cf9e7e8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:23.083053 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e04f7-664c-4883-b3ed-96f66cf9e7e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:23.084804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e04f7-664c-4883-b3ed-96f66cf9e7e8 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:23.094723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 447/900] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Aug 3 18:38:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:23.094723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e04f7-664c-4883-b3ed-96f66cf9e7e8 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:23.109474 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:23.142748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 454/901] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:23.145696 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb623e77-2092-402f-ace5-9bcd65c4d28d None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:23.215969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e04f7-664c-4883-b3ed-96f66cf9e7e8 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:23.254025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb623e77-2092-402f-ace5-9bcd65c4d28d None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:23.254985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092533d-75b1-455d-815c-996209e17755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:23.255632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092533d-75b1-455d-815c-996209e17755 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:23.255920 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092533d-75b1-455d-815c-996209e17755 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:23.257299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092533d-75b1-455d-815c-996209e17755 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:23.264635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092533d-75b1-455d-815c-996209e17755 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:23.266081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092533d-75b1-455d-815c-996209e17755 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-3092533d-75b1-455d-815c-996209e17755 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:38:23.278719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:23.279695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:23.281074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:23.281074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:23.281074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:38:23.281074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:38:23.281074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:38:23.281074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:38:23.281074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:38:23.281074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:38:23.281074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 18:38:23.281074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:23.281074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 448/902] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Aug 3 18:38:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:23.281074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092533d-75b1-455d-815c-996209e17755 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:23.300902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd873ab-136f-4054-8906-7830b89c4b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:23.302242 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd873ab-136f-4054-8906-7830b89c4b53 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:23.302765 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd873ab-136f-4054-8906-7830b89c4b53 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:23.303342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd873ab-136f-4054-8906-7830b89c4b53 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:23.304517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd873ab-136f-4054-8906-7830b89c4b53 None admin] RBAC: Adding query filter params (interface=public, service_id=4bad2de1dc534131a1e1624333dda1a9, region_id=RegionOne) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:23.305549 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd873ab-136f-4054-8906-7830b89c4b53 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:23.307873 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd873ab-136f-4054-8906-7830b89c4b53 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:23.316044 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 455/903] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Aug 3 18:38:23 2022] GET /identity/v3/endpoints?service_id=4bad2de1dc534131a1e1624333dda1a9&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:23.317076 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd873ab-136f-4054-8906-7830b89c4b53 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:23.335703 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:23.361474 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:23.386521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092533d-75b1-455d-815c-996209e17755 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:23.421931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108ed62-f687-4319-86a0-eeaa1253a7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:23.422801 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108ed62-f687-4319-86a0-eeaa1253a7a5 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:23.423092 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108ed62-f687-4319-86a0-eeaa1253a7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:23.423351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108ed62-f687-4319-86a0-eeaa1253a7a5 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:23.424037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108ed62-f687-4319-86a0-eeaa1253a7a5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:23.425421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108ed62-f687-4319-86a0-eeaa1253a7a5 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:23.435933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd873ab-136f-4054-8906-7830b89c4b53 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:23.444486 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 449/904] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Aug 3 18:38:23 2022] POST /identity/v3/services => generated 241 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:38:23.444486 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108ed62-f687-4319-86a0-eeaa1253a7a5 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:23.492468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a190eb7-e35d-4c39-bac5-c38b4e00c29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:23.493750 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a190eb7-e35d-4c39-bac5-c38b4e00c29a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:23.494189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a190eb7-e35d-4c39-bac5-c38b4e00c29a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:23.494613 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a190eb7-e35d-4c39-bac5-c38b4e00c29a None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:23.495815 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a190eb7-e35d-4c39-bac5-c38b4e00c29a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:23.497205 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a190eb7-e35d-4c39-bac5-c38b4e00c29a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:23.499077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a190eb7-e35d-4c39-bac5-c38b4e00c29a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:23.532176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 456/905] 10.222.0.69 () {56 vars in 1149 bytes} [Wed Aug 3 18:38:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:23.532955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a190eb7-e35d-4c39-bac5-c38b4e00c29a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:23.593580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108ed62-f687-4319-86a0-eeaa1253a7a5 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:23.608846 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:23.641063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bb106-4135-4724-a365-e1a224ca8510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:23.641966 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bb106-4135-4724-a365-e1a224ca8510 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:23.642398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bb106-4135-4724-a365-e1a224ca8510 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:23.642820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bb106-4135-4724-a365-e1a224ca8510 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:23.643675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961bb106-4135-4724-a365-e1a224ca8510 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:23.644345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:23.645191 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961bb106-4135-4724-a365-e1a224ca8510 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:23.652414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 450/906] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Aug 3 18:38:23 2022] GET /identity/v3/services => generated 1497 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:38:23.657384 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a190eb7-e35d-4c39-bac5-c38b4e00c29a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:23.703475 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e2f83-cf82-48a2-bceb-1d23edd065e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:23.704124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e2f83-cf82-48a2-bceb-1d23edd065e1 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:23.704433 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e2f83-cf82-48a2-bceb-1d23edd065e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:23.704707 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e2f83-cf82-48a2-bceb-1d23edd065e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:23.788852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e2f83-cf82-48a2-bceb-1d23edd065e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:23.790847 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e2f83-cf82-48a2-bceb-1d23edd065e1 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:23.888681 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:23.901330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 457/907] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:38:23 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:38:23.908319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bb106-4135-4724-a365-e1a224ca8510 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:24.028515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bb106-4135-4724-a365-e1a224ca8510 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:24.066564 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff67c1-4a8a-4ee3-bda0-c7ef3141fb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:24.067258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff67c1-4a8a-4ee3-bda0-c7ef3141fb77 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:24.067539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff67c1-4a8a-4ee3-bda0-c7ef3141fb77 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:24.067868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff67c1-4a8a-4ee3-bda0-c7ef3141fb77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:24.080974 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff67c1-4a8a-4ee3-bda0-c7ef3141fb77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:24.082428 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff67c1-4a8a-4ee3-bda0-c7ef3141fb77 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:24.098599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-dbff67c1-4a8a-4ee3-bda0-c7ef3141fb77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:24.098599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:24.098599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:24.098599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:24.098599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:24.098599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:24.098599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:24.098599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:24.098599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:24.098599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:24.098599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:24.098599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:24.098599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:24.098599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:24.098599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:24.099568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:24.101299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:24.101299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:24.101299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:24.101299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:24.101299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 451/908] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:24.107494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e2f83-cf82-48a2-bceb-1d23edd065e1 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:24.216986 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e2f83-cf82-48a2-bceb-1d23edd065e1 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:24.254582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f92e10-a59e-411d-9409-e29f56e7a113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:24.255210 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f92e10-a59e-411d-9409-e29f56e7a113 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:24.255496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f92e10-a59e-411d-9409-e29f56e7a113 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:24.255771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f92e10-a59e-411d-9409-e29f56e7a113 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:24.257106 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f92e10-a59e-411d-9409-e29f56e7a113 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:24.257578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f92e10-a59e-411d-9409-e29f56e7a113 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:24.258963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f92e10-a59e-411d-9409-e29f56e7a113 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:24.271546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 458/909] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:24.277579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff67c1-4a8a-4ee3-bda0-c7ef3141fb77 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:24.312862 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:24.392001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff67c1-4a8a-4ee3-bda0-c7ef3141fb77 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:24.429779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387800f-a4ed-4ede-9298-6ec8d33d384a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:24.430905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387800f-a4ed-4ede-9298-6ec8d33d384a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:24.431388 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387800f-a4ed-4ede-9298-6ec8d33d384a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:24.431879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387800f-a4ed-4ede-9298-6ec8d33d384a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:24.439216 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387800f-a4ed-4ede-9298-6ec8d33d384a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:24.440955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387800f-a4ed-4ede-9298-6ec8d33d384a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:24.453890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-a387800f-a4ed-4ede-9298-6ec8d33d384a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:24.453890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:24.453890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:24.453890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:24.453890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:24.453890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:24.453890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:24.453890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:24.453890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:24.453890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:24.453890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:24.453890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:24.453890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:24.453890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:24.453890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:24.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:24.455775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:24.455775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:24.455775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:24.455775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:24.455775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:24.455775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:24.455775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:24.455775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:24.458355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 452/910] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:24.464379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f92e10-a59e-411d-9409-e29f56e7a113 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:24.479959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd058ff0-33cf-435b-9e64-cf8a494e9c02 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:24.480314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd058ff0-33cf-435b-9e64-cf8a494e9c02 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:24.480533 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd058ff0-33cf-435b-9e64-cf8a494e9c02 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:24.482023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 453/911] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:38:24.487084 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060e791-abd3-4e5c-8a7a-9708f8c0267d None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:24.487608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060e791-abd3-4e5c-8a7a-9708f8c0267d None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:24.488082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060e791-abd3-4e5c-8a7a-9708f8c0267d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:24.509514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-c060e791-abd3-4e5c-8a7a-9708f8c0267d None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:24.511057 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c060e791-abd3-4e5c-8a7a-9708f8c0267d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:24.531513 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:24.588615 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 454/912] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:24.596573 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f92e10-a59e-411d-9409-e29f56e7a113 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:24.606181 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060e791-abd3-4e5c-8a7a-9708f8c0267d None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:24.628631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:24.642021 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fe2ee-2f15-4792-ad79-8eb3b8b72b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:24.642982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fe2ee-2f15-4792-ad79-8eb3b8b72b8f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:24.643514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fe2ee-2f15-4792-ad79-8eb3b8b72b8f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:24.644007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fe2ee-2f15-4792-ad79-8eb3b8b72b8f None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:24.644997 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73fe2ee-2f15-4792-ad79-8eb3b8b72b8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:24.645509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73fe2ee-2f15-4792-ad79-8eb3b8b72b8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:24.647711 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73fe2ee-2f15-4792-ad79-8eb3b8b72b8f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:24.670792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 459/913] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:38:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:24.671228 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fe2ee-2f15-4792-ad79-8eb3b8b72b8f None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:24.727725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060e791-abd3-4e5c-8a7a-9708f8c0267d None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:24.767298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01ab53-e904-492c-9072-8d311957bb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:24.767994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01ab53-e904-492c-9072-8d311957bb45 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:24.768280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01ab53-e904-492c-9072-8d311957bb45 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:24.768589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01ab53-e904-492c-9072-8d311957bb45 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:24.769317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca01ab53-e904-492c-9072-8d311957bb45 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:24.770794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca01ab53-e904-492c-9072-8d311957bb45 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:24.779786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fe2ee-2f15-4792-ad79-8eb3b8b72b8f None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-ca01ab53-e904-492c-9072-8d311957bb45 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:24.781874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:24.782962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:24.783849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:24.783849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:24.783849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:24.783849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:24.783849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:24.783849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:24.783849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:24.783849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:24.783849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:24.783849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:24.783849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:24.783849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:24.783849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 18:38:24.783849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:24.783849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 455/914] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:38:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:24.791862 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01ab53-e904-492c-9072-8d311957bb45 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:24.797983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:24.818237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28a19b-ce93-43da-9163-57aaad6abb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:24.818904 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28a19b-ce93-43da-9163-57aaad6abb1f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:24.819187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28a19b-ce93-43da-9163-57aaad6abb1f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:24.819466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28a19b-ce93-43da-9163-57aaad6abb1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:24.820264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28a19b-ce93-43da-9163-57aaad6abb1f None admin] RBAC: Adding query filter params (role.id=6dcc297751a745d38cb3a184f49cfd60, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=6cebfb2bb96942c9b64720d9b8d9c6b2) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:24.820606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28a19b-ce93-43da-9163-57aaad6abb1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:24.822052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28a19b-ce93-43da-9163-57aaad6abb1f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:24.827955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 460/915] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:38:24 2022] GET /identity/v3/role_assignments?user.id=6cebfb2bb96942c9b64720d9b8d9c6b2&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=6dcc297751a745d38cb3a184f49cfd60 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:24.905122 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:24.942369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01ab53-e904-492c-9072-8d311957bb45 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:24.984948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e505ec9-bd86-4d7c-8dba-288ac091f857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:24.985848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e505ec9-bd86-4d7c-8dba-288ac091f857 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:24.986284 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e505ec9-bd86-4d7c-8dba-288ac091f857 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:24.986714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e505ec9-bd86-4d7c-8dba-288ac091f857 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:24.987553 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e505ec9-bd86-4d7c-8dba-288ac091f857 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:24.988016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e505ec9-bd86-4d7c-8dba-288ac091f857 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:24.989550 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e505ec9-bd86-4d7c-8dba-288ac091f857 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:24.996578 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 456/916] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:38:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:25.001943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28a19b-ce93-43da-9163-57aaad6abb1f None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:25.113269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28a19b-ce93-43da-9163-57aaad6abb1f None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:25.150100 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee9e50-3034-4816-9944-7faeaa4f800d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:25.150728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee9e50-3034-4816-9944-7faeaa4f800d None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:25.151007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee9e50-3034-4816-9944-7faeaa4f800d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:25.151278 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee9e50-3034-4816-9944-7faeaa4f800d None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:25.151957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee9e50-3034-4816-9944-7faeaa4f800d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:25.152258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee9e50-3034-4816-9944-7faeaa4f800d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:25.153651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee9e50-3034-4816-9944-7faeaa4f800d None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:25.159110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 461/917] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Aug 3 18:38:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:25.164588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e505ec9-bd86-4d7c-8dba-288ac091f857 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:25.269487 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e505ec9-bd86-4d7c-8dba-288ac091f857 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:25.311032 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b87fc-7a91-4f2a-aa5c-71de9838dcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:25.311694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b87fc-7a91-4f2a-aa5c-71de9838dcc2 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:25.311979 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b87fc-7a91-4f2a-aa5c-71de9838dcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:25.312257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b87fc-7a91-4f2a-aa5c-71de9838dcc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:25.313019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b87fc-7a91-4f2a-aa5c-71de9838dcc2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:25.314390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b87fc-7a91-4f2a-aa5c-71de9838dcc2 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:25.330511 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:25.336554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 457/918] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:38:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:38:25.675894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b8c00-899e-4598-90e5-e4eaafb48f11 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:25.676958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b8c00-899e-4598-90e5-e4eaafb48f11 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:25.677468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b8c00-899e-4598-90e5-e4eaafb48f11 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:25.679852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 462/919] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:25.685572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbed299-7f1e-473c-a06c-8070b94459e9 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:25.686023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbed299-7f1e-473c-a06c-8070b94459e9 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:25.686114 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbed299-7f1e-473c-a06c-8070b94459e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:25.706089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-8dbed299-7f1e-473c-a06c-8070b94459e9 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:25.707299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8dbed299-7f1e-473c-a06c-8070b94459e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:25.788774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 458/920] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:25.812444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b8c00-899e-4598-90e5-e4eaafb48f11 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:25.861270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f5c74-1607-44fe-91f2-fd76ed357928 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:25.861579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f5c74-1607-44fe-91f2-fd76ed357928 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:25.861841 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f5c74-1607-44fe-91f2-fd76ed357928 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:25.863330 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 459/921] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:38:25.869107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21ee34-c88a-4266-ae86-25f7b21a544a None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:25.869407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21ee34-c88a-4266-ae86-25f7b21a544a None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:25.869664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21ee34-c88a-4266-ae86-25f7b21a544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:25.876682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:25.889844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-da21ee34-c88a-4266-ae86-25f7b21a544a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:25.891113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-da21ee34-c88a-4266-ae86-25f7b21a544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:25.954620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b8c00-899e-4598-90e5-e4eaafb48f11 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:25.974165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 460/922] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:25.995819 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21ee34-c88a-4266-ae86-25f7b21a544a None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:26.003974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1848586-b26e-48b3-b697-ce183c3eea13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:26.004979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1848586-b26e-48b3-b697-ce183c3eea13 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:26.005505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1848586-b26e-48b3-b697-ce183c3eea13 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:26.006782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1848586-b26e-48b3-b697-ce183c3eea13 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:26.007689 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1848586-b26e-48b3-b697-ce183c3eea13 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:26.009820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1848586-b26e-48b3-b697-ce183c3eea13 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-c1848586-b26e-48b3-b697-ce183c3eea13 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:26.020177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:26.021353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:26.022455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:26.022455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:26.022455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:26.022455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:26.022455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:26.022455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:26.022455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:26.022455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:26.022455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:26.022455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:26.022455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:26.022455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:26.022455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 18:38:26.022455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:26.025684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 463/923] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Aug 3 18:38:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:26.032344 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1848586-b26e-48b3-b697-ce183c3eea13 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:26.067079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:26.121151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21ee34-c88a-4266-ae86-25f7b21a544a None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:26.158601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473d393-33a1-400c-a11d-ac8a887ea7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:26.159294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473d393-33a1-400c-a11d-ac8a887ea7d1 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:26.159574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473d393-33a1-400c-a11d-ac8a887ea7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:26.159895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473d393-33a1-400c-a11d-ac8a887ea7d1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:26.160756 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473d393-33a1-400c-a11d-ac8a887ea7d1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:26.161947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473d393-33a1-400c-a11d-ac8a887ea7d1 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-3473d393-33a1-400c-a11d-ac8a887ea7d1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:26.171770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:26.173000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:26.173865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 461/924] 10.222.0.69 () {56 vars in 1132 bytes} [Wed Aug 3 18:38:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:26.177315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1848586-b26e-48b3-b697-ce183c3eea13 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:26.179742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:26.182236 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473d393-33a1-400c-a11d-ac8a887ea7d1 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:26.225984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc947-cb3b-4dd7-99e8-d672efb56a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:26.226893 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc947-cb3b-4dd7-99e8-d672efb56a7d None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:26.227336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc947-cb3b-4dd7-99e8-d672efb56a7d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:26.227764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc947-cb3b-4dd7-99e8-d672efb56a7d None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:26.228646 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cc947-cb3b-4dd7-99e8-d672efb56a7d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:26.229121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cc947-cb3b-4dd7-99e8-d672efb56a7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:26.231179 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cc947-cb3b-4dd7-99e8-d672efb56a7d None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:26.242428 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 464/925] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Aug 3 18:38:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:26.244595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc947-cb3b-4dd7-99e8-d672efb56a7d None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:26.313607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473d393-33a1-400c-a11d-ac8a887ea7d1 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:26.340583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:26.359293 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff8407-bd80-4e49-8698-fe7472894197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:26.360570 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff8407-bd80-4e49-8698-fe7472894197 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:26.361266 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff8407-bd80-4e49-8698-fe7472894197 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:26.361712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff8407-bd80-4e49-8698-fe7472894197 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:26.362795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff8407-bd80-4e49-8698-fe7472894197 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:26.363663 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff8407-bd80-4e49-8698-fe7472894197 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:26.366104 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff8407-bd80-4e49-8698-fe7472894197 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:26.367076 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc947-cb3b-4dd7-99e8-d672efb56a7d None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:26.373019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 462/926] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Aug 3 18:38:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:26.378870 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff8407-bd80-4e49-8698-fe7472894197 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:26.415538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ffdca-db20-44b4-98b9-02f1b3d22650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:26.416362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ffdca-db20-44b4-98b9-02f1b3d22650 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:26.416751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ffdca-db20-44b4-98b9-02f1b3d22650 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:26.417201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ffdca-db20-44b4-98b9-02f1b3d22650 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:26.417912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ffdca-db20-44b4-98b9-02f1b3d22650 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:26.418542 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ffdca-db20-44b4-98b9-02f1b3d22650 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:26.419810 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ffdca-db20-44b4-98b9-02f1b3d22650 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:26.431037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 465/927] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Aug 3 18:38:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:26.432436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ffdca-db20-44b4-98b9-02f1b3d22650 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:26.469962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:26.527810 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff8407-bd80-4e49-8698-fe7472894197 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:26.554861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ffdca-db20-44b4-98b9-02f1b3d22650 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:26.573965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85801f-818c-4cef-90a9-43e12eba1227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:26.574890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85801f-818c-4cef-90a9-43e12eba1227 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:26.575175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85801f-818c-4cef-90a9-43e12eba1227 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:26.575449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85801f-818c-4cef-90a9-43e12eba1227 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:26.576138 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85801f-818c-4cef-90a9-43e12eba1227 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:26.576473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85801f-818c-4cef-90a9-43e12eba1227 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:26.578290 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85801f-818c-4cef-90a9-43e12eba1227 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:26.585590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 463/928] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Aug 3 18:38:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:26.590770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85801f-818c-4cef-90a9-43e12eba1227 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:26.597054 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb6513-451e-41f8-9ef6-b4da7bdcf4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:26.597683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb6513-451e-41f8-9ef6-b4da7bdcf4af None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:26.597963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb6513-451e-41f8-9ef6-b4da7bdcf4af None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:26.598234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb6513-451e-41f8-9ef6-b4da7bdcf4af None admin] PATH_INFO: `/v3/endpoints` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:26.599084 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb6513-451e-41f8-9ef6-b4da7bdcf4af None admin] RBAC: Adding query filter params (interface=public, service_id=7c7bfec65e5a4851b4f0ba2a7782f2b6, region_id=RegionOne) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:26.599390 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb6513-451e-41f8-9ef6-b4da7bdcf4af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:26.601641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb6513-451e-41f8-9ef6-b4da7bdcf4af None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:26.607530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 466/929] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Aug 3 18:38:26 2022] GET /identity/v3/endpoints?service_id=7c7bfec65e5a4851b4f0ba2a7782f2b6&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:26.613209 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb6513-451e-41f8-9ef6-b4da7bdcf4af None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:26.665254 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:26.721422 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:26.724167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85801f-818c-4cef-90a9-43e12eba1227 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:26.734182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb6513-451e-41f8-9ef6-b4da7bdcf4af None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:26.776816 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5901b-99c0-4e1b-acbf-6d029d8779b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:26.777409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5901b-99c0-4e1b-acbf-6d029d8779b8 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:26.777693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5901b-99c0-4e1b-acbf-6d029d8779b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:26.778026 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5901b-99c0-4e1b-acbf-6d029d8779b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:26.778794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5901b-99c0-4e1b-acbf-6d029d8779b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:26.779438 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb74fbb-a985-42b5-b9e5-c169fc6e246c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:26.780168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5901b-99c0-4e1b-acbf-6d029d8779b8 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:26.780601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb74fbb-a985-42b5-b9e5-c169fc6e246c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:26.781100 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb74fbb-a985-42b5-b9e5-c169fc6e246c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:26.781572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb74fbb-a985-42b5-b9e5-c169fc6e246c None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:26.782597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb74fbb-a985-42b5-b9e5-c169fc6e246c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:26.784352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb74fbb-a985-42b5-b9e5-c169fc6e246c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:26.792957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 467/930] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Aug 3 18:38:26 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:38:26.818432 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 464/931] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:38:26 2022] POST /identity/v3/endpoints => generated 347 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:38:27.030874 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6147d3-c33c-4f69-8c6e-4cf2ea97280f None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:27.031070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6147d3-c33c-4f69-8c6e-4cf2ea97280f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:27.031349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6147d3-c33c-4f69-8c6e-4cf2ea97280f None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:27.033066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 468/932] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:27.038720 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36371d-c3d0-4954-a851-2b47365cfb85 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:27.039267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36371d-c3d0-4954-a851-2b47365cfb85 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:27.039971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36371d-c3d0-4954-a851-2b47365cfb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:27.060728 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-2a36371d-c3d0-4954-a851-2b47365cfb85 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:27.061920 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-2a36371d-c3d0-4954-a851-2b47365cfb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:27.146041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 465/933] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:27.168726 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6147d3-c33c-4f69-8c6e-4cf2ea97280f None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:27.282095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6147d3-c33c-4f69-8c6e-4cf2ea97280f None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:27.320813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830acb7-4d91-4504-8bfb-65b0e568b001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:27.321528 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830acb7-4d91-4504-8bfb-65b0e568b001 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:27.321813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830acb7-4d91-4504-8bfb-65b0e568b001 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:27.322086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830acb7-4d91-4504-8bfb-65b0e568b001 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:27.332781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830acb7-4d91-4504-8bfb-65b0e568b001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-3830acb7-4d91-4504-8bfb-65b0e568b001 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:27.338133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:27.339110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:27.340397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:27.340397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:27.340397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:27.340397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:27.340397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:27.340397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:27.340397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:27.340397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:27.340397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:27.340397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:27.340397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 18:38:27.340397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:27.340397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 469/934] 10.222.0.69 () {56 vars in 1120 bytes} [Wed Aug 3 18:38:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:27.345507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36371d-c3d0-4954-a851-2b47365cfb85 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:27.377133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:27.453782 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36371d-c3d0-4954-a851-2b47365cfb85 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:27.490744 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6485ac8-d57d-4fe8-9354-3dc1ae2d19cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:27.492090 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6485ac8-d57d-4fe8-9354-3dc1ae2d19cd None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:27.492642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6485ac8-d57d-4fe8-9354-3dc1ae2d19cd None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:27.493185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6485ac8-d57d-4fe8-9354-3dc1ae2d19cd None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:27.494223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6485ac8-d57d-4fe8-9354-3dc1ae2d19cd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:27.494724 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6485ac8-d57d-4fe8-9354-3dc1ae2d19cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:27.496280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6485ac8-d57d-4fe8-9354-3dc1ae2d19cd None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:27.505836 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 466/935] 10.222.0.69 () {56 vars in 1115 bytes} [Wed Aug 3 18:38:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:27.512038 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830acb7-4d91-4504-8bfb-65b0e568b001 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:27.533071 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:27.542510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458a40b-426c-4d0f-88d3-ef834de9f61f None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:27.542797 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458a40b-426c-4d0f-88d3-ef834de9f61f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:27.543077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458a40b-426c-4d0f-88d3-ef834de9f61f None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:27.544690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 467/936] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:27.549519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8daefe7-d8c6-4324-9f60-387539d20e76 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:27.549802 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8daefe7-d8c6-4324-9f60-387539d20e76 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:27.550080 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8daefe7-d8c6-4324-9f60-387539d20e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:27.572568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-d8daefe7-d8c6-4324-9f60-387539d20e76 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:27.574068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d8daefe7-d8c6-4324-9f60-387539d20e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:27.628364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830acb7-4d91-4504-8bfb-65b0e568b001 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:27.666560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7870a35-6d0d-4b42-9754-3ad17aee46b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:27.667417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7870a35-6d0d-4b42-9754-3ad17aee46b8 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:27.667818 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7870a35-6d0d-4b42-9754-3ad17aee46b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:27.668194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7870a35-6d0d-4b42-9754-3ad17aee46b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:27.673361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 468/937] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:27.681981 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7870a35-6d0d-4b42-9754-3ad17aee46b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:27.683539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7870a35-6d0d-4b42-9754-3ad17aee46b8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:27.694470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8daefe7-d8c6-4324-9f60-387539d20e76 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:27.703788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-d7870a35-6d0d-4b42-9754-3ad17aee46b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:27.703788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:27.703788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:27.703788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:27.703788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:27.703788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:27.703788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:27.703788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:27.703788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:27.703788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:27.703788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:27.703788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:27.703788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:27.703788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:27.703788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:27.705845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:27.708001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:27.708001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:27.708001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:27.708001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:27.708001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 470/938] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:27.714988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7870a35-6d0d-4b42-9754-3ad17aee46b8 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:27.729889 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:27.808972 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8daefe7-d8c6-4324-9f60-387539d20e76 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:27.845684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa1f5c-7d80-4e16-a43d-67688f0e88e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:27.846476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa1f5c-7d80-4e16-a43d-67688f0e88e1 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:27.846761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa1f5c-7d80-4e16-a43d-67688f0e88e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:27.847025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa1f5c-7d80-4e16-a43d-67688f0e88e1 None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:27.848032 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa1f5c-7d80-4e16-a43d-67688f0e88e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:27.849808 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa1f5c-7d80-4e16-a43d-67688f0e88e1 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:27.853299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: WARNING keystone.server.flask.common [None req-86fa1f5c-7d80-4e16-a43d-67688f0e88e1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 03 18:38:27.872778 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:27.874451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.sql.core [None req-86fa1f5c-7d80-4e16-a43d-67688f0e88e1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 03 18:38:27.874451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 03 18:38:27.874451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [parameters: {'id': '732015063e474f3783609c58d78ed8b3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 03 18:38:27.874451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91894) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 03 18:38:27.898074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-86fa1f5c-7d80-4e16-a43d-67688f0e88e1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 732015063e474f3783609c58d78ed8b3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 732015063e474f3783609c58d78ed8b3. Aug 03 18:38:27.898074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:27.898074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 18:38:27.898074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 18:38:27.898074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 18:38:27.898074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 18:38:27.898074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 18:38:27.898074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 18:38:27.898074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 18:38:27.898074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application conn.query(q) Aug 03 18:38:27.898074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 18:38:27.898074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 18:38:27.898074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application result.read() Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 03 18:38:27.899848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application next(self.gen) Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application yield resource Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application next(self.gen) Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self.session.rollback() Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self.force_reraise() Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise self.value Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application session.commit() Aug 03 18:38:27.901973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._prepare_impl() Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self.session.flush() Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._flush(objects) Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application compat.raise_( Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application flush_context.execute() Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 03 18:38:27.903779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rec.execute(self) Aug 03 18:38:27.905607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 03 18:38:27.905607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 03 18:38:27.905607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 03 18:38:27.905607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 03 18:38:27.905607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 03 18:38:27.905607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 03 18:38:27.905607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 03 18:38:27.905607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 03 18:38:27.905607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 03 18:38:27.905607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 03 18:38:27.905607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 03 18:38:27.905607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 03 18:38:27.905607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 03 18:38:27.905607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 03 18:38:27.905607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application conn.query(q) Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application result.read() Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 18:38:27.932878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 18:38:27.934985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 18:38:27.934985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 18:38:27.934985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 18:38:27.934985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 18:38:27.934985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 18:38:27.934985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 18:38:27.934985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 03 18:38:27.934985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 03 18:38:27.934985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [parameters: {'id': '732015063e474f3783609c58d78ed8b3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 03 18:38:27.934985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 03 18:38:27.934985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:27.934985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:27.934985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:27.934985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:27.934985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:27.936851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:27.938960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:27.938960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 03 18:38:27.938960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 03 18:38:27.938960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:27.938960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:27.938960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 03 18:38:27.938960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 03 18:38:27.938960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 732015063e474f3783609c58d78ed8b3. Aug 03 18:38:27.938960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:27.938960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 469/939] 10.222.0.69 () {60 vars in 1140 bytes} [Wed Aug 3 18:38:27 2022] POST /identity/v3/projects => generated 262 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:27.938960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa1f5c-7d80-4e16-a43d-67688f0e88e1 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:27.956555 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7870a35-6d0d-4b42-9754-3ad17aee46b8 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:27.966257 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05398dc2-d887-42c0-863a-226f956d0d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:27.967270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05398dc2-d887-42c0-863a-226f956d0d41 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:27.967552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05398dc2-d887-42c0-863a-226f956d0d41 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:27.968025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05398dc2-d887-42c0-863a-226f956d0d41 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:27.968949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05398dc2-d887-42c0-863a-226f956d0d41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:27.969259 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05398dc2-d887-42c0-863a-226f956d0d41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:27.970836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05398dc2-d887-42c0-863a-226f956d0d41 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:27.988401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 471/940] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:27.995513 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05398dc2-d887-42c0-863a-226f956d0d41 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:28.038081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:28.063417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa1f5c-7d80-4e16-a43d-67688f0e88e1 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:28.102999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4203a3-7932-49e5-9d56-f1a33e71082b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:28.103675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4203a3-7932-49e5-9d56-f1a33e71082b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:28.103956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4203a3-7932-49e5-9d56-f1a33e71082b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:28.104225 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4203a3-7932-49e5-9d56-f1a33e71082b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:28.111452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4203a3-7932-49e5-9d56-f1a33e71082b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:28.113188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4203a3-7932-49e5-9d56-f1a33e71082b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:28.126328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-9b4203a3-7932-49e5-9d56-f1a33e71082b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:28.126328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:28.126328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:28.126328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:28.126328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:28.126328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:28.126328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:28.126328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:28.126328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:28.126328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:28.126328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:28.126328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:28.126328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:28.126328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:28.126328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:28.127314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:28.128510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:28.128510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:28.128510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:28.128510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:28.128510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:28.128510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:28.128510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:28.128510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:28.132748 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 470/941] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:28.135151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4203a3-7932-49e5-9d56-f1a33e71082b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:28.140868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05398dc2-d887-42c0-863a-226f956d0d41 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:28.177118 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:28.184908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972f7c7-1d37-44ae-b865-c9972318ba72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:28.185940 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972f7c7-1d37-44ae-b865-c9972318ba72 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:28.186430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972f7c7-1d37-44ae-b865-c9972318ba72 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:28.186885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972f7c7-1d37-44ae-b865-c9972318ba72 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:28.195809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972f7c7-1d37-44ae-b865-c9972318ba72 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:28.197937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972f7c7-1d37-44ae-b865-c9972318ba72 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-d972f7c7-1d37-44ae-b865-c9972318ba72 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:38:28.212240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:28.213327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:28.214317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:28.214317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:28.214317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:38:28.214317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:38:28.214317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:38:28.214317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:38:28.214317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:38:28.214317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:38:28.214317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 18:38:28.214317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:28.218466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 472/942] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Aug 3 18:38:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:28.219611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972f7c7-1d37-44ae-b865-c9972318ba72 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:28.267572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4203a3-7932-49e5-9d56-f1a33e71082b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:28.302630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b734f-e0ac-432b-a615-4f1ddfc4381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:28.303825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b734f-e0ac-432b-a615-4f1ddfc4381f None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:28.304455 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b734f-e0ac-432b-a615-4f1ddfc4381f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:28.304455 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b734f-e0ac-432b-a615-4f1ddfc4381f None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:28.305223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b734f-e0ac-432b-a615-4f1ddfc4381f None admin] RBAC: Adding query filter params (name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:28.305532 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b734f-e0ac-432b-a615-4f1ddfc4381f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:28.306978 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b734f-e0ac-432b-a615-4f1ddfc4381f None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:28.318282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:28.336382 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 471/943] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Aug 3 18:38:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:28.365053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972f7c7-1d37-44ae-b865-c9972318ba72 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:28.406264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b29083-2b7f-4905-9a89-3adaaf37c058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:28.406906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b29083-2b7f-4905-9a89-3adaaf37c058 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:28.407248 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b29083-2b7f-4905-9a89-3adaaf37c058 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:28.407527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b29083-2b7f-4905-9a89-3adaaf37c058 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:28.408437 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b29083-2b7f-4905-9a89-3adaaf37c058 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:28.408750 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b29083-2b7f-4905-9a89-3adaaf37c058 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:28.410266 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b29083-2b7f-4905-9a89-3adaaf37c058 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:28.432321 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 473/944] 10.222.0.69 () {56 vars in 1149 bytes} [Wed Aug 3 18:38:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:28.438484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b734f-e0ac-432b-a615-4f1ddfc4381f None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:28.558357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b734f-e0ac-432b-a615-4f1ddfc4381f None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:28.596085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54635f7-c804-4594-ae05-f6541e54e2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:28.596677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54635f7-c804-4594-ae05-f6541e54e2a3 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:28.596960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54635f7-c804-4594-ae05-f6541e54e2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:28.597230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54635f7-c804-4594-ae05-f6541e54e2a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:28.609718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54635f7-c804-4594-ae05-f6541e54e2a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:28.611180 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54635f7-c804-4594-ae05-f6541e54e2a3 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:28.628172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-e54635f7-c804-4594-ae05-f6541e54e2a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:28.628172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:28.628172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:28.628172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:28.628172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:28.628172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:28.628172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:28.628172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:28.628172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:28.628172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:28.628172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:28.628172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:28.628172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:28.628172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:28.628172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:28.629452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:28.630900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:28.630900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:28.630900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:28.630900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:28.630900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 472/945] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:28.636841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b29083-2b7f-4905-9a89-3adaaf37c058 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:28.692735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:28.749405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b29083-2b7f-4905-9a89-3adaaf37c058 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:28.788115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8142a-83e4-4012-9167-a9aa62d075da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:28.788776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8142a-83e4-4012-9167-a9aa62d075da None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:28.789065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8142a-83e4-4012-9167-a9aa62d075da None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:28.789348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8142a-83e4-4012-9167-a9aa62d075da None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:28.790063 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8142a-83e4-4012-9167-a9aa62d075da None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:28.790373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8142a-83e4-4012-9167-a9aa62d075da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:28.791764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8142a-83e4-4012-9167-a9aa62d075da None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:28.805312 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 474/946] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:28.810527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54635f7-c804-4594-ae05-f6541e54e2a3 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:28.864423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:28.930938 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54635f7-c804-4594-ae05-f6541e54e2a3 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:28.946420 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436eec2-4bcd-4613-a652-282ec0fba5fa None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:28.946974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436eec2-4bcd-4613-a652-282ec0fba5fa None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:28.946974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436eec2-4bcd-4613-a652-282ec0fba5fa None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:28.949266 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 475/947] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:38:28.954522 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9fe2c-544a-453a-ac57-3b99b4792de2 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:28.954814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9fe2c-544a-453a-ac57-3b99b4792de2 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:28.955087 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9fe2c-544a-453a-ac57-3b99b4792de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:28.975696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e30b33-c2c3-4b19-9232-de9f3eeebc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:28.976490 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e30b33-c2c3-4b19-9232-de9f3eeebc3c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:28.976971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e30b33-c2c3-4b19-9232-de9f3eeebc3c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:28.977352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e30b33-c2c3-4b19-9232-de9f3eeebc3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:28.977537 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d8a9fe2c-544a-453a-ac57-3b99b4792de2 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:28.979290 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d8a9fe2c-544a-453a-ac57-3b99b4792de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:28.989404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e30b33-c2c3-4b19-9232-de9f3eeebc3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:28.991515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e30b33-c2c3-4b19-9232-de9f3eeebc3c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:29.011478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-05e30b33-c2c3-4b19-9232-de9f3eeebc3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:29.011478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:29.011478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:29.011478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:29.011478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:29.011478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:29.011478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:29.011478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:29.011478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:29.011478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:29.011478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:29.011478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:29.011478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:29.011478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:29.011478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:29.012478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:29.013398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:29.013398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:29.013398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:29.013398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:29.013398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:29.013398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:29.013398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:29.013398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:29.013398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 473/948] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:29.019130 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e30b33-c2c3-4b19-9232-de9f3eeebc3c None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:29.076585 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 476/949] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:29.095924 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9fe2c-544a-453a-ac57-3b99b4792de2 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:29.126664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e30b33-c2c3-4b19-9232-de9f3eeebc3c None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:29.138528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:29.164005 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:29.166729 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78619f6-fdd5-4926-94ef-681fd91370bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:29.167660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78619f6-fdd5-4926-94ef-681fd91370bd None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:29.167952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78619f6-fdd5-4926-94ef-681fd91370bd None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:29.168310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78619f6-fdd5-4926-94ef-681fd91370bd None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:29.169100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78619f6-fdd5-4926-94ef-681fd91370bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:29.169465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78619f6-fdd5-4926-94ef-681fd91370bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:29.170995 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78619f6-fdd5-4926-94ef-681fd91370bd None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:29.191564 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 474/950] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:38:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:29.191564 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b72871-ab3d-4d06-9045-67a6d4b3c428 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:29.191564 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b72871-ab3d-4d06-9045-67a6d4b3c428 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:29.191564 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b72871-ab3d-4d06-9045-67a6d4b3c428 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:29.191991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 475/951] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:29.193381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b72871-ab3d-4d06-9045-67a6d4b3c428 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:29.214681 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9fe2c-544a-453a-ac57-3b99b4792de2 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:29.253930 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe96b3-24f2-4b7f-b0fe-ae927281cd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:29.254598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe96b3-24f2-4b7f-b0fe-ae927281cd77 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:29.254881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe96b3-24f2-4b7f-b0fe-ae927281cd77 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:29.255159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe96b3-24f2-4b7f-b0fe-ae927281cd77 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:29.255800 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe96b3-24f2-4b7f-b0fe-ae927281cd77 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:29.258364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe96b3-24f2-4b7f-b0fe-ae927281cd77 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-56fe96b3-24f2-4b7f-b0fe-ae927281cd77 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:29.267774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:29.268845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:29.269680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:29.269680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:29.269680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:29.269680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:29.269680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:29.269680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:29.269680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:29.269680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:29.269680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:29.269680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:29.269680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:29.269680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:29.269680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 18:38:29.269680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:29.269680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 477/952] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Aug 3 18:38:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:29.271055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51b13f-48eb-47aa-abf7-9245eb302cb0 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:29.271055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51b13f-48eb-47aa-abf7-9245eb302cb0 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:29.271230 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51b13f-48eb-47aa-abf7-9245eb302cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:29.292865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-0a51b13f-48eb-47aa-abf7-9245eb302cb0 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:29.295057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0a51b13f-48eb-47aa-abf7-9245eb302cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:29.313980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b72871-ab3d-4d06-9045-67a6d4b3c428 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:29.362088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462268c4-9487-460b-b223-31237782407e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:29.362765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462268c4-9487-460b-b223-31237782407e None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:29.363050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462268c4-9487-460b-b223-31237782407e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:29.363404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462268c4-9487-460b-b223-31237782407e None admin] PATH_INFO: `/v3/projects/59f00389bb404b4392d98e22e045c84f/users/6cebfb2bb96942c9b64720d9b8d9c6b2/roles/6dcc297751a745d38cb3a184f49cfd60` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:29.366365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462268c4-9487-460b-b223-31237782407e None admin] RBAC: Authorizing `identity:create_grant(project_id=59f00389bb404b4392d98e22e045c84f, user_id=6cebfb2bb96942c9b64720d9b8d9c6b2, role_id=6dcc297751a745d38cb3a184f49cfd60)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:29.367909 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462268c4-9487-460b-b223-31237782407e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:29.377465 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 478/953] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:29.378471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51b13f-48eb-47aa-abf7-9245eb302cb0 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:29.388923 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 476/954] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:38:29 2022] PUT /identity/v3/projects/59f00389bb404b4392d98e22e045c84f/users/6cebfb2bb96942c9b64720d9b8d9c6b2/roles/6dcc297751a745d38cb3a184f49cfd60 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:38:29.405029 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462268c4-9487-460b-b223-31237782407e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:29.426698 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:29.449601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:29.532972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51b13f-48eb-47aa-abf7-9245eb302cb0 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:29.545408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462268c4-9487-460b-b223-31237782407e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:29.574449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73c187-c20c-4250-9cc7-94d158c2eda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:29.575235 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73c187-c20c-4250-9cc7-94d158c2eda2 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:29.575613 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73c187-c20c-4250-9cc7-94d158c2eda2 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:29.575984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73c187-c20c-4250-9cc7-94d158c2eda2 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:29.577637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73c187-c20c-4250-9cc7-94d158c2eda2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:29.577637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73c187-c20c-4250-9cc7-94d158c2eda2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:29.578703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73c187-c20c-4250-9cc7-94d158c2eda2 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:29.583898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 479/955] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Aug 3 18:38:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:29.589396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b4255-f45e-4d72-9f29-1e3c79657cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:29.590069 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b4255-f45e-4d72-9f29-1e3c79657cb0 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:29.590192 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73c187-c20c-4250-9cc7-94d158c2eda2 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:29.590355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b4255-f45e-4d72-9f29-1e3c79657cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:29.590626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b4255-f45e-4d72-9f29-1e3c79657cb0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:29.591275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b4255-f45e-4d72-9f29-1e3c79657cb0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:29.592674 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b4255-f45e-4d72-9f29-1e3c79657cb0 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-4b1b4255-f45e-4d72-9f29-1e3c79657cb0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:29.608168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:29.609638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:29.610732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:29.610732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:29.610732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:29.610732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:29.610732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:29.610732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:29.610732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:29.610732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:29.610732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:29.610732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:29.610732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:29.610732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:29.610732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 18:38:29.610732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:29.613853 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 477/956] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:29.620859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b4255-f45e-4d72-9f29-1e3c79657cb0 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:29.708698 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:29.712927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73c187-c20c-4250-9cc7-94d158c2eda2 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:29.738764 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:29.749540 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b4255-f45e-4d72-9f29-1e3c79657cb0 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:29.763748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313febc-ecb0-4068-827e-f8bb79f7e1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:29.764496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313febc-ecb0-4068-827e-f8bb79f7e1c2 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:29.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313febc-ecb0-4068-827e-f8bb79f7e1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:29.765424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313febc-ecb0-4068-827e-f8bb79f7e1c2 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:29.766203 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7313febc-ecb0-4068-827e-f8bb79f7e1c2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:29.767042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7313febc-ecb0-4068-827e-f8bb79f7e1c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:29.768795 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7313febc-ecb0-4068-827e-f8bb79f7e1c2 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:29.780377 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 480/957] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Aug 3 18:38:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:29.782350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313febc-ecb0-4068-827e-f8bb79f7e1c2 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:29.794828 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfaa18-42e2-42b9-84d0-bf4023748515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:29.795481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfaa18-42e2-42b9-84d0-bf4023748515 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:29.795760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfaa18-42e2-42b9-84d0-bf4023748515 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:29.796037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfaa18-42e2-42b9-84d0-bf4023748515 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:29.796753 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfaa18-42e2-42b9-84d0-bf4023748515 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:29.797058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfaa18-42e2-42b9-84d0-bf4023748515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:29.798413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfaa18-42e2-42b9-84d0-bf4023748515 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:29.810577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 478/958] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Aug 3 18:38:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:29.811652 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfaa18-42e2-42b9-84d0-bf4023748515 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:29.908317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313febc-ecb0-4068-827e-f8bb79f7e1c2 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:29.948632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfaa18-42e2-42b9-84d0-bf4023748515 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:29.951242 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b1ea2-60e7-4689-a859-a418ce9372cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:29.952515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b1ea2-60e7-4689-a859-a418ce9372cb None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:29.953039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b1ea2-60e7-4689-a859-a418ce9372cb None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:29.953551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b1ea2-60e7-4689-a859-a418ce9372cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:29.954530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b1ea2-60e7-4689-a859-a418ce9372cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:29.956176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b1ea2-60e7-4689-a859-a418ce9372cb None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:29.982792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 481/959] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:38:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:38:29.999382 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1ad06-8aa3-4f7e-ba02-0b8e1beeefa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:30.000059 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1ad06-8aa3-4f7e-ba02-0b8e1beeefa8 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:30.000372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1ad06-8aa3-4f7e-ba02-0b8e1beeefa8 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:30.000650 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1ad06-8aa3-4f7e-ba02-0b8e1beeefa8 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:30.001476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1ad06-8aa3-4f7e-ba02-0b8e1beeefa8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:30.001784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1ad06-8aa3-4f7e-ba02-0b8e1beeefa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:30.003568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1ad06-8aa3-4f7e-ba02-0b8e1beeefa8 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:30.009764 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 479/960] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Aug 3 18:38:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:30.606863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47820ba-cb54-4a51-9bbe-a7659291fbe4 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:30.607199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47820ba-cb54-4a51-9bbe-a7659291fbe4 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:30.607452 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47820ba-cb54-4a51-9bbe-a7659291fbe4 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:30.609103 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 482/961] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:30.614135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36082025-ea2d-471d-a5a4-04c87aae1ddc None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:30.614425 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36082025-ea2d-471d-a5a4-04c87aae1ddc None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:30.614702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36082025-ea2d-471d-a5a4-04c87aae1ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:30.635741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-36082025-ea2d-471d-a5a4-04c87aae1ddc None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:30.637345 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-36082025-ea2d-471d-a5a4-04c87aae1ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:30.653235 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:30.717773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 480/962] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:30.738842 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47820ba-cb54-4a51-9bbe-a7659291fbe4 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:30.762283 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:30.862017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47820ba-cb54-4a51-9bbe-a7659291fbe4 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:30.900615 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ecfd7-621c-4c6f-98a1-e00ee6e90c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:30.901566 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ecfd7-621c-4c6f-98a1-e00ee6e90c26 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:30.902099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ecfd7-621c-4c6f-98a1-e00ee6e90c26 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:30.902578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ecfd7-621c-4c6f-98a1-e00ee6e90c26 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:30.903466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ecfd7-621c-4c6f-98a1-e00ee6e90c26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:30.905198 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ecfd7-621c-4c6f-98a1-e00ee6e90c26 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:30.907970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: WARNING keystone.server.flask.common [None req-166ecfd7-621c-4c6f-98a1-e00ee6e90c26 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 03 18:38:30.921769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.sql.core [None req-166ecfd7-621c-4c6f-98a1-e00ee6e90c26 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 03 18:38:30.921769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 03 18:38:30.921769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [parameters: {'user_id': 'a51ab1b2435b4e7eb272842b93f90dc1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 03 18:38:30.921769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91895) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 03 18:38:30.930277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-166ecfd7-621c-4c6f-98a1-e00ee6e90c26 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 03 18:38:30.930277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:30.930277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 18:38:30.930277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 18:38:30.930277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 18:38:30.930277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 18:38:30.930277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 18:38:30.930277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 18:38:30.930277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 18:38:30.930277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application conn.query(q) Aug 03 18:38:30.930277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 18:38:30.930277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 18:38:30.930277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 18:38:30.930277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application result.read() Aug 03 18:38:30.930277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 18:38:30.931199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application next(self.gen) Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application yield resource Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application next(self.gen) Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self.session.rollback() Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self.force_reraise() Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise self.value Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application session.commit() Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 03 18:38:30.932186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._prepare_impl() Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self.session.flush() Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._flush(objects) Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application compat.raise_( Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application flush_context.execute() Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rec.execute(self) Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 03 18:38:30.933611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 18:38:30.935684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application conn.query(q) Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application result.read() Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 18:38:30.937826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 18:38:30.939944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 18:38:30.939944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 18:38:30.939944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 18:38:30.939944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 18:38:30.939944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 03 18:38:30.939944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 03 18:38:30.939944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a51ab1b2435b4e7eb272842b93f90dc1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 03 18:38:30.939944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 03 18:38:30.939944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:30.939944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:30.939944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:30.939944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:30.939944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:30.939944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:30.939944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 03 18:38:30.941996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 03 18:38:30.945643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 03 18:38:30.945643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 03 18:38:30.945643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 03 18:38:30.945643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 03 18:38:30.945643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 03 18:38:30.945643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:30.947819 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 483/963] 10.222.0.69 () {60 vars in 1131 bytes} [Wed Aug 3 18:38:30 2022] POST /identity/v3/users => generated 160 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:30.953694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36082025-ea2d-471d-a5a4-04c87aae1ddc None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:31.062772 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36082025-ea2d-471d-a5a4-04c87aae1ddc None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:31.101288 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779eb27e-e6db-4ead-b914-97ef6d7f617c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:31.102023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779eb27e-e6db-4ead-b914-97ef6d7f617c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:31.102329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779eb27e-e6db-4ead-b914-97ef6d7f617c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:31.102601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779eb27e-e6db-4ead-b914-97ef6d7f617c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:31.110527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779eb27e-e6db-4ead-b914-97ef6d7f617c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:31.112085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779eb27e-e6db-4ead-b914-97ef6d7f617c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-779eb27e-e6db-4ead-b914-97ef6d7f617c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:38:31.126267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:31.128766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:31.131398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:31.131398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:31.131398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:38:31.131398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:38:31.131398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:38:31.131398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:38:31.131398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:38:31.131398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:38:31.131398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 18:38:31.131398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:31.132728 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 481/964] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:38:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:31.136310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ecfd7-621c-4c6f-98a1-e00ee6e90c26 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:31.230527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:31.257704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ecfd7-621c-4c6f-98a1-e00ee6e90c26 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:31.294689 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe1b11-2da4-4a3c-af7e-fa9005610d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:31.295726 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe1b11-2da4-4a3c-af7e-fa9005610d05 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:31.296231 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe1b11-2da4-4a3c-af7e-fa9005610d05 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:31.296753 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe1b11-2da4-4a3c-af7e-fa9005610d05 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:31.297723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe1b11-2da4-4a3c-af7e-fa9005610d05 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:31.298234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe1b11-2da4-4a3c-af7e-fa9005610d05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:31.300091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe1b11-2da4-4a3c-af7e-fa9005610d05 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:31.341478 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 484/965] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Aug 3 18:38:31 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:31.587267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260befe5-cdf1-43fa-b52e-3f8560979b8e None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:31.587911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260befe5-cdf1-43fa-b52e-3f8560979b8e None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:31.588447 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260befe5-cdf1-43fa-b52e-3f8560979b8e None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:31.590116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 482/966] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:31.595979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411d185-5214-47ba-a90c-c8826cd03d42 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:31.596268 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411d185-5214-47ba-a90c-c8826cd03d42 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:31.596572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411d185-5214-47ba-a90c-c8826cd03d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:31.615820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-4411d185-5214-47ba-a90c-c8826cd03d42 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:31.617535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4411d185-5214-47ba-a90c-c8826cd03d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:31.689034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 485/967] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:31.712518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260befe5-cdf1-43fa-b52e-3f8560979b8e None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:31.723645 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:31.819567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260befe5-cdf1-43fa-b52e-3f8560979b8e None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:31.854176 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e78147-3233-4ca4-8f83-e7bac488c476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:31.854733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e78147-3233-4ca4-8f83-e7bac488c476 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:31.855010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e78147-3233-4ca4-8f83-e7bac488c476 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:31.855278 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e78147-3233-4ca4-8f83-e7bac488c476 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:31.866039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e78147-3233-4ca4-8f83-e7bac488c476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-b3e78147-3233-4ca4-8f83-e7bac488c476 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:31.870914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:31.871857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:31.873070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:31.873070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:31.873070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:31.873070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:31.873070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:31.873070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:31.873070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:31.873070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:31.873070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:31.873070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:31.873070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 18:38:31.873070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:31.876373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 483/968] 10.222.0.69 () {56 vars in 1120 bytes} [Wed Aug 3 18:38:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:31.878479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411d185-5214-47ba-a90c-c8826cd03d42 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:31.958120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:31.997053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411d185-5214-47ba-a90c-c8826cd03d42 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:32.035625 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7c1df-67b5-42f4-89e9-9c9347aef254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:32.036246 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7c1df-67b5-42f4-89e9-9c9347aef254 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:32.036554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7c1df-67b5-42f4-89e9-9c9347aef254 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:32.036828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7c1df-67b5-42f4-89e9-9c9347aef254 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:32.037763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7c1df-67b5-42f4-89e9-9c9347aef254 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:32.038065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7c1df-67b5-42f4-89e9-9c9347aef254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:32.039466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7c1df-67b5-42f4-89e9-9c9347aef254 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:32.052354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 486/969] 10.222.0.69 () {56 vars in 1115 bytes} [Wed Aug 3 18:38:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:32.054089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e78147-3233-4ca4-8f83-e7bac488c476 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:32.072792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6834e0-306c-47a5-abb7-a3e5cf26b4a5 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:32.073124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6834e0-306c-47a5-abb7-a3e5cf26b4a5 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:32.073366 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6834e0-306c-47a5-abb7-a3e5cf26b4a5 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:32.074762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 487/970] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:38:32.080629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b84951-b92a-4980-a782-3da8705641e6 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:32.080916 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b84951-b92a-4980-a782-3da8705641e6 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:32.081224 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b84951-b92a-4980-a782-3da8705641e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:32.100537 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-a5b84951-b92a-4980-a782-3da8705641e6 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:32.101737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a5b84951-b92a-4980-a782-3da8705641e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:32.125926 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:32.161025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e78147-3233-4ca4-8f83-e7bac488c476 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:32.170997 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 488/971] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:32.172256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f3ab8-50d3-4b3d-adc0-4acb6d244ee5 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:32.172575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f3ab8-50d3-4b3d-adc0-4acb6d244ee5 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:32.172853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f3ab8-50d3-4b3d-adc0-4acb6d244ee5 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:32.174349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 489/972] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:32.180143 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413500d-86fd-42ac-b459-59664248fc58 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:32.180465 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413500d-86fd-42ac-b459-59664248fc58 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:32.180748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413500d-86fd-42ac-b459-59664248fc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:32.199189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73443b68-2c60-4d27-98c5-fe72ebf0b177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:32.199490 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-2413500d-86fd-42ac-b459-59664248fc58 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:32.199842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73443b68-2c60-4d27-98c5-fe72ebf0b177 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:32.200127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73443b68-2c60-4d27-98c5-fe72ebf0b177 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:32.200443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73443b68-2c60-4d27-98c5-fe72ebf0b177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:32.200723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2413500d-86fd-42ac-b459-59664248fc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:32.211420 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73443b68-2c60-4d27-98c5-fe72ebf0b177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:32.212916 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73443b68-2c60-4d27-98c5-fe72ebf0b177 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:32.227465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-73443b68-2c60-4d27-98c5-fe72ebf0b177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:32.227465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:32.227465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:32.227465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:32.227465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:32.227465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:32.227465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:32.227465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:32.227465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:32.227465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:32.227465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:32.227465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:32.227465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:32.227465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:32.227465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:32.228424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:32.229389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:32.229389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:32.229389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:32.229389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:32.229389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 484/973] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:32.234040 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73443b68-2c60-4d27-98c5-fe72ebf0b177 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:32.255160 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:32.270002 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 490/974] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:32.280533 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413500d-86fd-42ac-b459-59664248fc58 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:32.338280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73443b68-2c60-4d27-98c5-fe72ebf0b177 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:32.357984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:32.373546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98d7ca-9b70-4833-9eb5-05ae939aaba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:32.374182 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98d7ca-9b70-4833-9eb5-05ae939aaba3 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:32.374424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413500d-86fd-42ac-b459-59664248fc58 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:32.374536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98d7ca-9b70-4833-9eb5-05ae939aaba3 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:32.374721 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98d7ca-9b70-4833-9eb5-05ae939aaba3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:32.375348 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98d7ca-9b70-4833-9eb5-05ae939aaba3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:32.376804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98d7ca-9b70-4833-9eb5-05ae939aaba3 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-6f98d7ca-9b70-4833-9eb5-05ae939aaba3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:32.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:32.387988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:32.388877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:32.388877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:32.388877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:32.388877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:32.388877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:32.388877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:32.388877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:32.388877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:32.388877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:32.388877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:32.388877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:32.388877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:32.388877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 18:38:32.388877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:32.388877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 485/975] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Aug 3 18:38:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:32.389697 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98d7ca-9b70-4833-9eb5-05ae939aaba3 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:32.414162 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca1f22-8bde-4a66-9ffb-7d23501668f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:32.414794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca1f22-8bde-4a66-9ffb-7d23501668f9 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:32.415077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca1f22-8bde-4a66-9ffb-7d23501668f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:32.415352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca1f22-8bde-4a66-9ffb-7d23501668f9 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:32.416041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca1f22-8bde-4a66-9ffb-7d23501668f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:32.416372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca1f22-8bde-4a66-9ffb-7d23501668f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:32.417739 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca1f22-8bde-4a66-9ffb-7d23501668f9 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:32.429622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 491/976] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:32.430480 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca1f22-8bde-4a66-9ffb-7d23501668f9 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:32.479117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:32.491677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98d7ca-9b70-4833-9eb5-05ae939aaba3 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:32.527158 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfa57f-a3f9-4a41-bf4a-b8698f564f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:32.527770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfa57f-a3f9-4a41-bf4a-b8698f564f48 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:32.528048 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfa57f-a3f9-4a41-bf4a-b8698f564f48 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:32.528412 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfa57f-a3f9-4a41-bf4a-b8698f564f48 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:32.529101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfa57f-a3f9-4a41-bf4a-b8698f564f48 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:32.530469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfa57f-a3f9-4a41-bf4a-b8698f564f48 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:32.532099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca1f22-8bde-4a66-9ffb-7d23501668f9 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:32.545786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 486/977] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Aug 3 18:38:32 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:38:32.546604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfa57f-a3f9-4a41-bf4a-b8698f564f48 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:32.582087 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6208d1-09e5-451a-a0ef-431cbd6e1c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:32.583041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6208d1-09e5-451a-a0ef-431cbd6e1c37 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:32.583325 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6208d1-09e5-451a-a0ef-431cbd6e1c37 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:32.583614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6208d1-09e5-451a-a0ef-431cbd6e1c37 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:32.584591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6208d1-09e5-451a-a0ef-431cbd6e1c37 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:32.585080 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6208d1-09e5-451a-a0ef-431cbd6e1c37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:32.586911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6208d1-09e5-451a-a0ef-431cbd6e1c37 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:32.597171 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 492/978] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Aug 3 18:38:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:32.598712 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6208d1-09e5-451a-a0ef-431cbd6e1c37 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:32.613666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:32.684319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfa57f-a3f9-4a41-bf4a-b8698f564f48 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:32.712182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6208d1-09e5-451a-a0ef-431cbd6e1c37 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:32.722853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:32.729510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dfe29-4205-4ee1-b9d7-4536c441d94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:32.730176 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dfe29-4205-4ee1-b9d7-4536c441d94a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:32.730457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dfe29-4205-4ee1-b9d7-4536c441d94a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:32.730727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dfe29-4205-4ee1-b9d7-4536c441d94a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:32.739837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01dfe29-4205-4ee1-b9d7-4536c441d94a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:32.742126 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01dfe29-4205-4ee1-b9d7-4536c441d94a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:32.751027 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645893b5-3999-45f9-a90d-25472912ac9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:32.751657 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645893b5-3999-45f9-a90d-25472912ac9e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:32.751944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645893b5-3999-45f9-a90d-25472912ac9e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:32.752226 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645893b5-3999-45f9-a90d-25472912ac9e None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:32.753293 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645893b5-3999-45f9-a90d-25472912ac9e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:32.753604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645893b5-3999-45f9-a90d-25472912ac9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-d01dfe29-4205-4ee1-b9d7-4536c441d94a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:38:32.754133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:32.755088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:32.756660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:32.756660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:32.756660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:38:32.756660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:38:32.756660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:38:32.756660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:38:32.756660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:38:32.756660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:38:32.756660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 18:38:32.756660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:32.757232 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645893b5-3999-45f9-a90d-25472912ac9e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:32.757346 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 487/979] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Aug 3 18:38:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:32.764337 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 493/980] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Aug 3 18:38:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:32.764515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dfe29-4205-4ee1-b9d7-4536c441d94a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:32.767271 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645893b5-3999-45f9-a90d-25472912ac9e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:32.877790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dfe29-4205-4ee1-b9d7-4536c441d94a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:32.880404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:32.894361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645893b5-3999-45f9-a90d-25472912ac9e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:32.925852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fb95c-8270-4984-9472-02ed95829192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:32.926563 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fb95c-8270-4984-9472-02ed95829192 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:32.926914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fb95c-8270-4984-9472-02ed95829192 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:32.927338 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fb95c-8270-4984-9472-02ed95829192 None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:32.931805 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b169d0b-1d2b-4143-8432-d2ba1459ff50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:32.932199 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fb95c-8270-4984-9472-02ed95829192 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:32.932641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fb95c-8270-4984-9472-02ed95829192 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:32.932991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b169d0b-1d2b-4143-8432-d2ba1459ff50 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:32.934012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b169d0b-1d2b-4143-8432-d2ba1459ff50 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:32.934303 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fb95c-8270-4984-9472-02ed95829192 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:32.934649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b169d0b-1d2b-4143-8432-d2ba1459ff50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:32.935575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b169d0b-1d2b-4143-8432-d2ba1459ff50 None admin] RBAC: Adding query filter params (interface=admin, service_id=7c7bfec65e5a4851b4f0ba2a7782f2b6, region_id=RegionOne) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:32.936364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b169d0b-1d2b-4143-8432-d2ba1459ff50 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:32.938900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b169d0b-1d2b-4143-8432-d2ba1459ff50 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:32.947759 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 494/981] 10.222.0.69 () {56 vars in 1249 bytes} [Wed Aug 3 18:38:32 2022] GET /identity/v3/endpoints?service_id=7c7bfec65e5a4851b4f0ba2a7782f2b6&interface=admin®ion_id=RegionOne => generated 193 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:32.952824 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b169d0b-1d2b-4143-8432-d2ba1459ff50 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:32.955229 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 488/982] 10.222.0.69 () {56 vars in 1149 bytes} [Wed Aug 3 18:38:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:32.962262 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fb95c-8270-4984-9472-02ed95829192 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:32.996751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:33.067871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b169d0b-1d2b-4143-8432-d2ba1459ff50 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:33.080869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fb95c-8270-4984-9472-02ed95829192 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:33.103876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b113c-89ff-4d55-a356-620a8b7d4c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:33.104534 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b113c-89ff-4d55-a356-620a8b7d4c22 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:33.104824 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b113c-89ff-4d55-a356-620a8b7d4c22 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:33.105094 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b113c-89ff-4d55-a356-620a8b7d4c22 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:33.105754 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b113c-89ff-4d55-a356-620a8b7d4c22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:33.107136 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b113c-89ff-4d55-a356-620a8b7d4c22 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:33.124316 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 495/983] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Aug 3 18:38:32 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:38:33.132328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c3b75-5f4a-4fac-a53d-325f0f5b6beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:33.133146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c3b75-5f4a-4fac-a53d-325f0f5b6beb None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:33.133524 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c3b75-5f4a-4fac-a53d-325f0f5b6beb None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:33.133894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c3b75-5f4a-4fac-a53d-325f0f5b6beb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:33.170693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:33.222522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c3b75-5f4a-4fac-a53d-325f0f5b6beb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:33.224153 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c3b75-5f4a-4fac-a53d-325f0f5b6beb None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:33.335933 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 489/984] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:38:32 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 18:38:33.341442 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b113c-89ff-4d55-a356-620a8b7d4c22 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:33.446660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:33.450211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b113c-89ff-4d55-a356-620a8b7d4c22 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:33.486650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d255d-2709-4018-b17f-779a42879392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:33.487216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d255d-2709-4018-b17f-779a42879392 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:33.487495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d255d-2709-4018-b17f-779a42879392 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:33.487769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d255d-2709-4018-b17f-779a42879392 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:33.499475 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d255d-2709-4018-b17f-779a42879392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:33.500950 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d255d-2709-4018-b17f-779a42879392 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:33.516392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-5b2d255d-2709-4018-b17f-779a42879392 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:33.516392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:33.516392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:33.516392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:33.516392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:33.516392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:33.516392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:33.516392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:33.516392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:33.516392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:33.516392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:33.516392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:33.516392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:33.516392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:33.516392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:33.517286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:33.518234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:33.518234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:33.518234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:33.518234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:33.518986 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 496/985] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:33.524530 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c3b75-5f4a-4fac-a53d-325f0f5b6beb None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:33.590301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:33.630589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c3b75-5f4a-4fac-a53d-325f0f5b6beb None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:33.667735 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6e645-cdb2-4c56-9d42-338d654db02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:33.668385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6e645-cdb2-4c56-9d42-338d654db02b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:33.668671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6e645-cdb2-4c56-9d42-338d654db02b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:33.668948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6e645-cdb2-4c56-9d42-338d654db02b None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:33.669644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6e645-cdb2-4c56-9d42-338d654db02b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:33.669949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6e645-cdb2-4c56-9d42-338d654db02b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:33.671306 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6e645-cdb2-4c56-9d42-338d654db02b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:33.688611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 490/986] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:33.689812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d255d-2709-4018-b17f-779a42879392 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:33.795156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d255d-2709-4018-b17f-779a42879392 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:33.829122 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4f41e-919f-4264-9a8d-a6fed94efb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:33.829769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4f41e-919f-4264-9a8d-a6fed94efb07 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:33.830052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4f41e-919f-4264-9a8d-a6fed94efb07 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:33.830325 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4f41e-919f-4264-9a8d-a6fed94efb07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:33.837930 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4f41e-919f-4264-9a8d-a6fed94efb07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:33.839383 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4f41e-919f-4264-9a8d-a6fed94efb07 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:33.851611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-03f4f41e-919f-4264-9a8d-a6fed94efb07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:33.851611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:33.851611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:33.851611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:33.851611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:33.851611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:33.851611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:33.851611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:33.851611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:33.851611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:33.851611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:33.851611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:33.851611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:33.851611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:33.851611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:33.852747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:33.854013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:33.854013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:33.854013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:33.854013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:33.854013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:33.854013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:33.854013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:33.854013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:33.854013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 497/987] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:33.860337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6e645-cdb2-4c56-9d42-338d654db02b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:33.973311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6e645-cdb2-4c56-9d42-338d654db02b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:33.997920 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:34.011234 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece81908-b4a5-43b0-8218-d9854266351b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:34.011860 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece81908-b4a5-43b0-8218-d9854266351b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:34.012145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece81908-b4a5-43b0-8218-d9854266351b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:34.012446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece81908-b4a5-43b0-8218-d9854266351b None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:34.013204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece81908-b4a5-43b0-8218-d9854266351b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:34.013510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece81908-b4a5-43b0-8218-d9854266351b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:34.014933 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece81908-b4a5-43b0-8218-d9854266351b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:34.031821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 491/988] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:38:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:34.034163 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4f41e-919f-4264-9a8d-a6fed94efb07 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:34.050884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:34.142924 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4f41e-919f-4264-9a8d-a6fed94efb07 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:34.178640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf67e71-c424-4b13-8ae0-77eb2c4218e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:34.179308 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf67e71-c424-4b13-8ae0-77eb2c4218e1 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:34.179600 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf67e71-c424-4b13-8ae0-77eb2c4218e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:34.179876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf67e71-c424-4b13-8ae0-77eb2c4218e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:34.180698 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf67e71-c424-4b13-8ae0-77eb2c4218e1 None admin] RBAC: Adding query filter params (role.id=6dcc297751a745d38cb3a184f49cfd60, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=6cebfb2bb96942c9b64720d9b8d9c6b2) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:34.181012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf67e71-c424-4b13-8ae0-77eb2c4218e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:34.182446 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf67e71-c424-4b13-8ae0-77eb2c4218e1 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:34.188900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 498/989] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:38:34 2022] GET /identity/v3/role_assignments?user.id=6cebfb2bb96942c9b64720d9b8d9c6b2&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=6dcc297751a745d38cb3a184f49cfd60 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:34.636884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a203963-fac6-4e8c-b4b3-2e5b8aa3c4bc None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:34.637413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a203963-fac6-4e8c-b4b3-2e5b8aa3c4bc None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:34.637413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a203963-fac6-4e8c-b4b3-2e5b8aa3c4bc None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:34.638802 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 492/990] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:34.643813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997a3cb-7377-4bab-9b1b-8b4b360a4516 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:34.644319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997a3cb-7377-4bab-9b1b-8b4b360a4516 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:34.644418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997a3cb-7377-4bab-9b1b-8b4b360a4516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:34.663456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-e997a3cb-7377-4bab-9b1b-8b4b360a4516 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:34.664659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e997a3cb-7377-4bab-9b1b-8b4b360a4516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:34.735091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 499/991] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:34.759379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a203963-fac6-4e8c-b4b3-2e5b8aa3c4bc None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:34.868926 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a203963-fac6-4e8c-b4b3-2e5b8aa3c4bc None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:34.904878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b432cd1-d339-4c34-8e23-56cb4903c75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:34.905442 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b432cd1-d339-4c34-8e23-56cb4903c75e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:34.905719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b432cd1-d339-4c34-8e23-56cb4903c75e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:34.905988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b432cd1-d339-4c34-8e23-56cb4903c75e None admin] PATH_INFO: `/v3/services/compute` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:34.906616 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b432cd1-d339-4c34-8e23-56cb4903c75e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:34.907963 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b432cd1-d339-4c34-8e23-56cb4903c75e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-1b432cd1-d339-4c34-8e23-56cb4903c75e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:34.918590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:34.919719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:34.925803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:34.925803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:34.925803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:34.925803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:34.925803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:34.925803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:34.925803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:34.925803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:34.925803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:34.925803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:34.925803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:34.925803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:34.925803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 18:38:34.925803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:34.925803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 493/992] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:34.926609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997a3cb-7377-4bab-9b1b-8b4b360a4516 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:34.935696 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:35.033163 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997a3cb-7377-4bab-9b1b-8b4b360a4516 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:35.068127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d1aa6-7836-43df-88fa-0ededf7fed5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:35.068838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d1aa6-7836-43df-88fa-0ededf7fed5a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:35.069119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d1aa6-7836-43df-88fa-0ededf7fed5a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:35.069397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d1aa6-7836-43df-88fa-0ededf7fed5a None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:35.070081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d1aa6-7836-43df-88fa-0ededf7fed5a None admin] RBAC: Adding query filter params (name=compute) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:35.070393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d1aa6-7836-43df-88fa-0ededf7fed5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:35.071783 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d1aa6-7836-43df-88fa-0ededf7fed5a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:35.081375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 500/993] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Aug 3 18:38:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:35.082608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b432cd1-d339-4c34-8e23-56cb4903c75e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:35.134070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:35.164568 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecba30-ce54-4b87-8910-6089a084016d None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:35.164756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecba30-ce54-4b87-8910-6089a084016d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:35.165042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecba30-ce54-4b87-8910-6089a084016d None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:35.167088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 501/994] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:35.172004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e75419-a830-462d-8ce9-2f1258690b2a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:35.172324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e75419-a830-462d-8ce9-2f1258690b2a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:35.172610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e75419-a830-462d-8ce9-2f1258690b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:35.192973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-76e75419-a830-462d-8ce9-2f1258690b2a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:35.194260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b432cd1-d339-4c34-8e23-56cb4903c75e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:35.194985 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-76e75419-a830-462d-8ce9-2f1258690b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:35.229710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6965d8b-1c74-47a2-bfa6-a40037ba9616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:35.230338 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6965d8b-1c74-47a2-bfa6-a40037ba9616 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:35.230620 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6965d8b-1c74-47a2-bfa6-a40037ba9616 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:35.230900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6965d8b-1c74-47a2-bfa6-a40037ba9616 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:35.231578 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6965d8b-1c74-47a2-bfa6-a40037ba9616 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:35.231889 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6965d8b-1c74-47a2-bfa6-a40037ba9616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:35.233282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6965d8b-1c74-47a2-bfa6-a40037ba9616 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:35.243739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 494/995] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Aug 3 18:38:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:35.244844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6965d8b-1c74-47a2-bfa6-a40037ba9616 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:35.271143 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:35.277298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 502/996] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:35.299512 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e75419-a830-462d-8ce9-2f1258690b2a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:35.366395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6965d8b-1c74-47a2-bfa6-a40037ba9616 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:35.378116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:35.408742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74973b7-9ba1-49b3-a041-00f6896ab92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:35.409416 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74973b7-9ba1-49b3-a041-00f6896ab92a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:35.409696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74973b7-9ba1-49b3-a041-00f6896ab92a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:35.409966 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74973b7-9ba1-49b3-a041-00f6896ab92a None admin] PATH_INFO: `/v3/endpoints` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:35.410931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74973b7-9ba1-49b3-a041-00f6896ab92a None admin] RBAC: Adding query filter params (interface=public, service_id=2098cf2ec5474a489b653c0e9263a329, region_id=RegionOne) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:35.411349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74973b7-9ba1-49b3-a041-00f6896ab92a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:35.413522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74973b7-9ba1-49b3-a041-00f6896ab92a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:35.414840 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e75419-a830-462d-8ce9-2f1258690b2a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:35.423511 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 495/997] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Aug 3 18:38:35 2022] GET /identity/v3/endpoints?service_id=2098cf2ec5474a489b653c0e9263a329&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:35.424565 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74973b7-9ba1-49b3-a041-00f6896ab92a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:35.454858 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d5423-fe10-4e2e-822f-b12fa8f86cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:35.455671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d5423-fe10-4e2e-822f-b12fa8f86cbe None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:35.456271 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d5423-fe10-4e2e-822f-b12fa8f86cbe None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:35.456587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d5423-fe10-4e2e-822f-b12fa8f86cbe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:35.457383 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d5423-fe10-4e2e-822f-b12fa8f86cbe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:35.459285 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d5423-fe10-4e2e-822f-b12fa8f86cbe None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-c52d5423-fe10-4e2e-822f-b12fa8f86cbe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:35.468560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:35.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:35.470859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:35.470859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:35.470859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:35.470859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:35.470859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:35.470859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:35.470859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:35.470859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:35.470859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:35.470859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:35.470859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:35.470859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:35.470859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 18:38:35.470859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:35.470859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 503/998] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Aug 3 18:38:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:35.476480 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d5423-fe10-4e2e-822f-b12fa8f86cbe None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:35.539469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74973b7-9ba1-49b3-a041-00f6896ab92a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:35.543864 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:35.577045 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e64fbe-a0e9-4479-bda4-982052a03eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:35.577729 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e64fbe-a0e9-4479-bda4-982052a03eda None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:35.578071 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e64fbe-a0e9-4479-bda4-982052a03eda None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:35.578357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e64fbe-a0e9-4479-bda4-982052a03eda None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:35.578989 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e64fbe-a0e9-4479-bda4-982052a03eda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:35.580387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e64fbe-a0e9-4479-bda4-982052a03eda None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:35.587112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d5423-fe10-4e2e-822f-b12fa8f86cbe None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:35.596319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 496/999] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Aug 3 18:38:35 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:38:35.635040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278254a-673d-4ce4-8d2d-5122725dbe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:35.635685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278254a-673d-4ce4-8d2d-5122725dbe15 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:35.635968 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278254a-673d-4ce4-8d2d-5122725dbe15 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:35.636243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278254a-673d-4ce4-8d2d-5122725dbe15 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:35.636967 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278254a-673d-4ce4-8d2d-5122725dbe15 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:35.637276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278254a-673d-4ce4-8d2d-5122725dbe15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:35.638684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278254a-673d-4ce4-8d2d-5122725dbe15 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:35.644655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 504/1000] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Aug 3 18:38:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:35.651289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e64fbe-a0e9-4479-bda4-982052a03eda None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:35.696215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:35.767189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e64fbe-a0e9-4479-bda4-982052a03eda None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:35.804819 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3cd88-c023-4ebc-a1f2-96b06d444f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:35.805447 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3cd88-c023-4ebc-a1f2-96b06d444f0c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:35.805734 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3cd88-c023-4ebc-a1f2-96b06d444f0c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:35.806010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3cd88-c023-4ebc-a1f2-96b06d444f0c None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:35.806698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3cd88-c023-4ebc-a1f2-96b06d444f0c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:35.807004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3cd88-c023-4ebc-a1f2-96b06d444f0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:35.808400 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3cd88-c023-4ebc-a1f2-96b06d444f0c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:35.813544 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 497/1001] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Aug 3 18:38:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:35.819602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278254a-673d-4ce4-8d2d-5122725dbe15 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:35.925338 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278254a-673d-4ce4-8d2d-5122725dbe15 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:35.958593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:35.969064 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271cd0d-ca86-4519-a0c9-4a679fe20ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:35.969862 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271cd0d-ca86-4519-a0c9-4a679fe20ffd None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:35.970249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271cd0d-ca86-4519-a0c9-4a679fe20ffd None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:35.970629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271cd0d-ca86-4519-a0c9-4a679fe20ffd None admin] PATH_INFO: `/v3/endpoints` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:35.971453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271cd0d-ca86-4519-a0c9-4a679fe20ffd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:35.972989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271cd0d-ca86-4519-a0c9-4a679fe20ffd None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:36.000317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 505/1002] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:38:35 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:38:36.292175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870ebc5-2d94-47d3-bf69-61a87d29ca62 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:36.292521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870ebc5-2d94-47d3-bf69-61a87d29ca62 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:36.292722 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870ebc5-2d94-47d3-bf69-61a87d29ca62 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:36.294289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 498/1003] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:36.299459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726bdb1-d11b-48d2-aeb0-d15bb04fe461 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:36.299756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726bdb1-d11b-48d2-aeb0-d15bb04fe461 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:36.300074 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726bdb1-d11b-48d2-aeb0-d15bb04fe461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:36.320799 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-4726bdb1-d11b-48d2-aeb0-d15bb04fe461 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:36.321628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4726bdb1-d11b-48d2-aeb0-d15bb04fe461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:36.398627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 506/1004] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:36 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:36.421480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870ebc5-2d94-47d3-bf69-61a87d29ca62 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:36.534285 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870ebc5-2d94-47d3-bf69-61a87d29ca62 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:36.535868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:36.571938 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac463a3-c5f3-43bf-9f62-f4cef99751be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:36.572553 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac463a3-c5f3-43bf-9f62-f4cef99751be None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:36.572836 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac463a3-c5f3-43bf-9f62-f4cef99751be None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:36.573221 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac463a3-c5f3-43bf-9f62-f4cef99751be None admin] PATH_INFO: `/v3/services/image` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:36.573789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac463a3-c5f3-43bf-9f62-f4cef99751be None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:36.575219 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac463a3-c5f3-43bf-9f62-f4cef99751be None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-eac463a3-c5f3-43bf-9f62-f4cef99751be None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:36.584719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:36.585749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:36.586657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:36.586657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:36.586657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:36.586657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:36.586657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:36.586657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:36.586657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:36.586657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:36.586657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:36.586657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:36.586657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:36.586657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:36.586657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 18:38:36.586657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:36.588677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 499/1005] 10.222.0.69 () {56 vars in 1105 bytes} [Wed Aug 3 18:38:36 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:36.591909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726bdb1-d11b-48d2-aeb0-d15bb04fe461 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:36.701209 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726bdb1-d11b-48d2-aeb0-d15bb04fe461 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:36.716730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:36.742430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae09195-5ca8-4cd0-8b0c-c9a63f1b7f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:36.743059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae09195-5ca8-4cd0-8b0c-c9a63f1b7f62 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:36.743343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae09195-5ca8-4cd0-8b0c-c9a63f1b7f62 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:36.743624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae09195-5ca8-4cd0-8b0c-c9a63f1b7f62 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:36.744342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae09195-5ca8-4cd0-8b0c-c9a63f1b7f62 None admin] RBAC: Adding query filter params (name=image) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:36.744655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae09195-5ca8-4cd0-8b0c-c9a63f1b7f62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:36.746546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae09195-5ca8-4cd0-8b0c-c9a63f1b7f62 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:36.756400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 507/1006] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:36.758667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac463a3-c5f3-43bf-9f62-f4cef99751be None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:36.863547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac463a3-c5f3-43bf-9f62-f4cef99751be None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:36.899565 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535161c0-4a4f-4583-b0fc-bd9b29f43f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:36.900201 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535161c0-4a4f-4583-b0fc-bd9b29f43f68 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:36.900514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535161c0-4a4f-4583-b0fc-bd9b29f43f68 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:36.900793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535161c0-4a4f-4583-b0fc-bd9b29f43f68 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:36.901490 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535161c0-4a4f-4583-b0fc-bd9b29f43f68 None admin] RBAC: Adding query filter params (type=image) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:36.901795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535161c0-4a4f-4583-b0fc-bd9b29f43f68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:36.903156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535161c0-4a4f-4583-b0fc-bd9b29f43f68 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:36.913162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 500/1007] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:37.640754 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f45c86-8a2e-4a21-bb94-2712235c8f09 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:37.641110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f45c86-8a2e-4a21-bb94-2712235c8f09 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:37.641308 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f45c86-8a2e-4a21-bb94-2712235c8f09 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:37.642801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 508/1008] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:37.648039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f209ae8-3722-492c-a78b-cc3e74dd1ec6 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:37.648352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f209ae8-3722-492c-a78b-cc3e74dd1ec6 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:37.648632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f209ae8-3722-492c-a78b-cc3e74dd1ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:37.666929 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-9f209ae8-3722-492c-a78b-cc3e74dd1ec6 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:37.668211 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9f209ae8-3722-492c-a78b-cc3e74dd1ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:37.694623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:37.738333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 501/1009] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:37 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:37.761791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f45c86-8a2e-4a21-bb94-2712235c8f09 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:37.869137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f45c86-8a2e-4a21-bb94-2712235c8f09 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:37.903443 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2ffe9-a394-45ac-a332-0eb1e22c5e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:37.904011 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2ffe9-a394-45ac-a332-0eb1e22c5e85 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:37.904373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2ffe9-a394-45ac-a332-0eb1e22c5e85 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:37.904611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2ffe9-a394-45ac-a332-0eb1e22c5e85 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:37.905238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2ffe9-a394-45ac-a332-0eb1e22c5e85 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:37.906603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2ffe9-a394-45ac-a332-0eb1e22c5e85 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-70d2ffe9-a394-45ac-a332-0eb1e22c5e85 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:37.917243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:37.918297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:37.919148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:37.919148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:37.919148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:37.919148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:37.919148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:37.919148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:37.919148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:37.919148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:37.919148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:37.919148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:37.919148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:37.919148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:37.919148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 18:38:37.919148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:37.925324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 509/1010] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:37.925451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f209ae8-3722-492c-a78b-cc3e74dd1ec6 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:38.034906 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f209ae8-3722-492c-a78b-cc3e74dd1ec6 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:38.062994 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03224a-7037-47be-b450-de86871c1b05 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:38.063275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03224a-7037-47be-b450-de86871c1b05 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:38.063551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03224a-7037-47be-b450-de86871c1b05 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:38.064934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 510/1011] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:38:38.070886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa64cf3-28f3-4546-b4cf-b4317d0317fa None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:38.071178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa64cf3-28f3-4546-b4cf-b4317d0317fa None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:38.071456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa64cf3-28f3-4546-b4cf-b4317d0317fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:38.080095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31141292-727d-4345-a5f4-0a31ca609968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:38.080945 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31141292-727d-4345-a5f4-0a31ca609968 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:38.081374 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31141292-727d-4345-a5f4-0a31ca609968 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:38.081788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31141292-727d-4345-a5f4-0a31ca609968 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:38.082637 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31141292-727d-4345-a5f4-0a31ca609968 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:38.083083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31141292-727d-4345-a5f4-0a31ca609968 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:38.085649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31141292-727d-4345-a5f4-0a31ca609968 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:38.090373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-5aa64cf3-28f3-4546-b4cf-b4317d0317fa None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:38.091260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 502/1012] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Aug 3 18:38:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:38.095911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5aa64cf3-28f3-4546-b4cf-b4317d0317fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:38.096985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31141292-727d-4345-a5f4-0a31ca609968 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:38.125125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:38.131599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:38.168190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 511/1013] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:38 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:38.191046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa64cf3-28f3-4546-b4cf-b4317d0317fa None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:38.209066 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31141292-727d-4345-a5f4-0a31ca609968 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:38.243986 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2feb3a-af55-4eed-a2ec-74601dc31392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:38.244639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2feb3a-af55-4eed-a2ec-74601dc31392 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:38.244922 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2feb3a-af55-4eed-a2ec-74601dc31392 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:38.245199 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2feb3a-af55-4eed-a2ec-74601dc31392 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:38.245877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2feb3a-af55-4eed-a2ec-74601dc31392 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:38.246182 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2feb3a-af55-4eed-a2ec-74601dc31392 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:38.247560 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2feb3a-af55-4eed-a2ec-74601dc31392 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:38.252656 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 503/1014] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Aug 3 18:38:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:38.259592 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2feb3a-af55-4eed-a2ec-74601dc31392 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:38.301620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa64cf3-28f3-4546-b4cf-b4317d0317fa None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:38.339438 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3542f-ec4d-44b8-9511-0763ab622bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:38.340088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3542f-ec4d-44b8-9511-0763ab622bb2 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:38.340409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3542f-ec4d-44b8-9511-0763ab622bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:38.340762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3542f-ec4d-44b8-9511-0763ab622bb2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:38.341955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3542f-ec4d-44b8-9511-0763ab622bb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:38.343394 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3542f-ec4d-44b8-9511-0763ab622bb2 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:38.345185 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 512/1015] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:38.351257 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3542f-ec4d-44b8-9511-0763ab622bb2 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:38.356071 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:38.369977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2feb3a-af55-4eed-a2ec-74601dc31392 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:38.397747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:38.408957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94216d58-51d2-4093-82f5-eaa21d36a7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:38.409588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94216d58-51d2-4093-82f5-eaa21d36a7cc None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:38.409875 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94216d58-51d2-4093-82f5-eaa21d36a7cc None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:38.410154 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94216d58-51d2-4093-82f5-eaa21d36a7cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:38.410881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94216d58-51d2-4093-82f5-eaa21d36a7cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:38.412570 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94216d58-51d2-4093-82f5-eaa21d36a7cc None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:38.444836 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 504/1016] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:38:38 2022] POST /identity/v3/endpoints => generated 334 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:38:38.473569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3542f-ec4d-44b8-9511-0763ab622bb2 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:38.523321 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4eaf3-72cc-4c7d-a8e5-70d0b387dd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:38.523962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4eaf3-72cc-4c7d-a8e5-70d0b387dd79 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:38.524241 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4eaf3-72cc-4c7d-a8e5-70d0b387dd79 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:38.524542 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4eaf3-72cc-4c7d-a8e5-70d0b387dd79 None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:38.525244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d4eaf3-72cc-4c7d-a8e5-70d0b387dd79 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:38.526699 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d4eaf3-72cc-4c7d-a8e5-70d0b387dd79 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:38.564328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 513/1017] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Aug 3 18:38:38 2022] POST /identity/v3/projects => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:38:38.951772 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a5165-c089-4524-8283-a01c9df7f557 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:38.952278 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a5165-c089-4524-8283-a01c9df7f557 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:38.952278 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a5165-c089-4524-8283-a01c9df7f557 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:38.953834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 505/1018] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:38.959580 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bb060-a3bb-458a-9b13-bef88ef4fd0a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:38.960026 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bb060-a3bb-458a-9b13-bef88ef4fd0a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:38.960332 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bb060-a3bb-458a-9b13-bef88ef4fd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:38.979506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-e54bb060-a3bb-458a-9b13-bef88ef4fd0a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:38.980730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e54bb060-a3bb-458a-9b13-bef88ef4fd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:39.051565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 514/1019] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:38 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:39.075000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a5165-c089-4524-8283-a01c9df7f557 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:39.181295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a5165-c089-4524-8283-a01c9df7f557 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:39.218087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d5ed3-5e2e-4b40-80ea-6844d02611e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:39.218718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d5ed3-5e2e-4b40-80ea-6844d02611e1 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:39.219002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d5ed3-5e2e-4b40-80ea-6844d02611e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:39.219279 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d5ed3-5e2e-4b40-80ea-6844d02611e1 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:39.219986 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d5ed3-5e2e-4b40-80ea-6844d02611e1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:39.221392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d5ed3-5e2e-4b40-80ea-6844d02611e1 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:39.244334 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 506/1020] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Aug 3 18:38:39 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:38:40.597203 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c98e57-c912-48d8-a097-080d693d086c None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:40.597792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c98e57-c912-48d8-a097-080d693d086c None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:40.597792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c98e57-c912-48d8-a097-080d693d086c None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:40.600339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 515/1021] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:40.604329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a62435-a579-4b84-a4f8-da4e0410f5c1 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:40.604756 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a62435-a579-4b84-a4f8-da4e0410f5c1 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:40.604938 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a62435-a579-4b84-a4f8-da4e0410f5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:40.612079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:40.628176 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-e4a62435-a579-4b84-a4f8-da4e0410f5c1 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:40.629406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e4a62435-a579-4b84-a4f8-da4e0410f5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:40.673328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26709818-a1cd-46b9-8809-b100105b779d None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:40.673616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26709818-a1cd-46b9-8809-b100105b779d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:40.673901 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26709818-a1cd-46b9-8809-b100105b779d None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:40.675262 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 516/1022] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:40.681329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef1559-2624-44bf-9efa-b4053268a9db None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:40.681623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef1559-2624-44bf-9efa-b4053268a9db None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:40.681900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef1559-2624-44bf-9efa-b4053268a9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:40.701257 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-5bef1559-2624-44bf-9efa-b4053268a9db None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:40.702464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5bef1559-2624-44bf-9efa-b4053268a9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:40.712429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 507/1023] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:40.716351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:40.733968 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a62435-a579-4b84-a4f8-da4e0410f5c1 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:40.781857 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 517/1024] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:40.807426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef1559-2624-44bf-9efa-b4053268a9db None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:40.838992 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a62435-a579-4b84-a4f8-da4e0410f5c1 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:40.869725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:40.876695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315660e4-fc3c-4450-989a-37291e8b343e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:40.877592 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315660e4-fc3c-4450-989a-37291e8b343e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:40.877876 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315660e4-fc3c-4450-989a-37291e8b343e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:40.878152 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315660e4-fc3c-4450-989a-37291e8b343e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:40.890055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315660e4-fc3c-4450-989a-37291e8b343e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-315660e4-fc3c-4450-989a-37291e8b343e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:40.895472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:40.897162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:40.898144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:40.898144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:40.898144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:40.898144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:40.898144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:40.898144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:40.898144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:40.898144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:40.898144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:40.898144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:40.898144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 18:38:40.898144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:40.902354 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 508/1025] 10.222.0.69 () {56 vars in 1120 bytes} [Wed Aug 3 18:38:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:40.902979 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315660e4-fc3c-4450-989a-37291e8b343e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:40.912265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef1559-2624-44bf-9efa-b4053268a9db None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:40.954718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa3a90-a7f3-4836-b6ab-15e56bab97eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:40.955371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa3a90-a7f3-4836-b6ab-15e56bab97eb None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:40.955651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa3a90-a7f3-4836-b6ab-15e56bab97eb None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:40.956052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa3a90-a7f3-4836-b6ab-15e56bab97eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:40.957239 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa3a90-a7f3-4836-b6ab-15e56bab97eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:40.959601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa3a90-a7f3-4836-b6ab-15e56bab97eb None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:40.961508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 518/1026] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:40.967122 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa3a90-a7f3-4836-b6ab-15e56bab97eb None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:40.985579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:41.020197 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315660e4-fc3c-4450-989a-37291e8b343e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:41.059644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825abca0-973c-46e0-8707-ad9211d46218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:41.060273 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825abca0-973c-46e0-8707-ad9211d46218 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:41.060583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825abca0-973c-46e0-8707-ad9211d46218 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:41.060859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825abca0-973c-46e0-8707-ad9211d46218 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:41.061588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825abca0-973c-46e0-8707-ad9211d46218 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:41.061895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825abca0-973c-46e0-8707-ad9211d46218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:41.063299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825abca0-973c-46e0-8707-ad9211d46218 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:41.073720 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 509/1027] 10.222.0.69 () {56 vars in 1115 bytes} [Wed Aug 3 18:38:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:41.078768 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa3a90-a7f3-4836-b6ab-15e56bab97eb None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:41.078992 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825abca0-973c-46e0-8707-ad9211d46218 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:41.117897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fd0c6-fd2b-410c-982f-13092a0f6595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:41.118534 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fd0c6-fd2b-410c-982f-13092a0f6595 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:41.118815 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fd0c6-fd2b-410c-982f-13092a0f6595 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:41.119095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fd0c6-fd2b-410c-982f-13092a0f6595 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:41.119854 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18fd0c6-fd2b-410c-982f-13092a0f6595 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:41.121408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18fd0c6-fd2b-410c-982f-13092a0f6595 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:41.144514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:41.147372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 519/1028] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:38:40 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:38:41.208114 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825abca0-973c-46e0-8707-ad9211d46218 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:41.247782 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a675a-e1eb-4200-bf7d-6b209af1ec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:41.248463 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a675a-e1eb-4200-bf7d-6b209af1ec23 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:41.248799 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a675a-e1eb-4200-bf7d-6b209af1ec23 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:41.249073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a675a-e1eb-4200-bf7d-6b209af1ec23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:41.262839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a675a-e1eb-4200-bf7d-6b209af1ec23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:41.264378 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a675a-e1eb-4200-bf7d-6b209af1ec23 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:41.280380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-920a675a-e1eb-4200-bf7d-6b209af1ec23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:41.280380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:41.280380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:41.280380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:41.280380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:41.280380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:41.280380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:41.280380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:41.280380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:41.280380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:41.280380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:41.280380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:41.280380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:41.280380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:41.280380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:41.281429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:41.282425 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:41.282425 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:41.282425 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:41.282425 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:41.284637 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 510/1029] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:41.287554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fd0c6-fd2b-410c-982f-13092a0f6595 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:41.288027 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67defe62-86cb-4f76-9118-813d84c53dd7 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:41.288352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67defe62-86cb-4f76-9118-813d84c53dd7 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:41.288640 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67defe62-86cb-4f76-9118-813d84c53dd7 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:41.290180 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 511/1030] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:38:41.295386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7ef5-4a17-41b0-985d-38b200a41e9b None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:41.295676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7ef5-4a17-41b0-985d-38b200a41e9b None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:41.295960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7ef5-4a17-41b0-985d-38b200a41e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:41.314882 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-058b7ef5-4a17-41b0-985d-38b200a41e9b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:41.316089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-058b7ef5-4a17-41b0-985d-38b200a41e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:41.379432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:41.395868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 512/1031] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:41.417020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fd0c6-fd2b-410c-982f-13092a0f6595 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:41.419522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b7ef5-4a17-41b0-985d-38b200a41e9b None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:41.454289 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed54e3a-09cb-4ca7-bd66-27260d4ae90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:41.454920 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed54e3a-09cb-4ca7-bd66-27260d4ae90f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:41.455202 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed54e3a-09cb-4ca7-bd66-27260d4ae90f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:41.455477 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed54e3a-09cb-4ca7-bd66-27260d4ae90f None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:41.456178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed54e3a-09cb-4ca7-bd66-27260d4ae90f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:41.456514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed54e3a-09cb-4ca7-bd66-27260d4ae90f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:41.457815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:41.457950 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed54e3a-09cb-4ca7-bd66-27260d4ae90f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:41.471306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 520/1032] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:41.476735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed54e3a-09cb-4ca7-bd66-27260d4ae90f None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:41.544506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b7ef5-4a17-41b0-985d-38b200a41e9b None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:41.582636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcb542-2c03-4e96-840a-141c6add165b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:41.584092 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcb542-2c03-4e96-840a-141c6add165b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:41.584483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcb542-2c03-4e96-840a-141c6add165b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:41.584765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcb542-2c03-4e96-840a-141c6add165b None admin] PATH_INFO: `/v3/services/image` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:41.585402 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcb542-2c03-4e96-840a-141c6add165b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:41.587111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcb542-2c03-4e96-840a-141c6add165b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:41.589964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed54e3a-09cb-4ca7-bd66-27260d4ae90f None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-6ebcb542-2c03-4e96-840a-141c6add165b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:41.596223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:41.597248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:41.598121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:41.598121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:41.598121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:41.598121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:41.598121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:41.598121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:41.598121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:41.598121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:41.598121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:41.598121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:41.598121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:41.598121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:41.598121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 18:38:41.598121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:41.598121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 513/1033] 10.222.0.69 () {56 vars in 1105 bytes} [Wed Aug 3 18:38:41 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:41.604233 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcb542-2c03-4e96-840a-141c6add165b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:41.625024 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:41.631524 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a19e86-3a41-4577-95e5-f8915333b1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:41.632101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a19e86-3a41-4577-95e5-f8915333b1c5 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:41.632418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a19e86-3a41-4577-95e5-f8915333b1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:41.632692 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a19e86-3a41-4577-95e5-f8915333b1c5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:41.640412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a19e86-3a41-4577-95e5-f8915333b1c5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:41.641856 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a19e86-3a41-4577-95e5-f8915333b1c5 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-94a19e86-3a41-4577-95e5-f8915333b1c5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:38:41.654267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:41.655280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:41.656393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:41.656393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:41.656393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:38:41.656393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:38:41.656393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:38:41.656393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:38:41.656393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:38:41.656393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:38:41.656393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:38:41.656393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:41.656393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 521/1034] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Aug 3 18:38:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:41.661209 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a19e86-3a41-4577-95e5-f8915333b1c5 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:41.717389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:41.721596 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcb542-2c03-4e96-840a-141c6add165b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:41.766725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14c3e6-12c4-4801-a51f-e9dfe60bd0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:41.767443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14c3e6-12c4-4801-a51f-e9dfe60bd0fe None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:41.767796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14c3e6-12c4-4801-a51f-e9dfe60bd0fe None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:41.768142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14c3e6-12c4-4801-a51f-e9dfe60bd0fe None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:41.768916 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14c3e6-12c4-4801-a51f-e9dfe60bd0fe None admin] RBAC: Adding query filter params (name=image) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:41.769290 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14c3e6-12c4-4801-a51f-e9dfe60bd0fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:41.770732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14c3e6-12c4-4801-a51f-e9dfe60bd0fe None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:41.776136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 514/1035] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:41.781974 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14c3e6-12c4-4801-a51f-e9dfe60bd0fe None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:41.791813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a19e86-3a41-4577-95e5-f8915333b1c5 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:41.828934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573afacd-4ce2-45a0-a6b4-ff238adeb50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:41.829553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573afacd-4ce2-45a0-a6b4-ff238adeb50d None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:41.829833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573afacd-4ce2-45a0-a6b4-ff238adeb50d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:41.830106 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573afacd-4ce2-45a0-a6b4-ff238adeb50d None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:41.830906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573afacd-4ce2-45a0-a6b4-ff238adeb50d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:41.831212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573afacd-4ce2-45a0-a6b4-ff238adeb50d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:41.832674 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573afacd-4ce2-45a0-a6b4-ff238adeb50d None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:41.851776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 522/1036] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Aug 3 18:38:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:41.857392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573afacd-4ce2-45a0-a6b4-ff238adeb50d None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:41.888341 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14c3e6-12c4-4801-a51f-e9dfe60bd0fe None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:41.911230 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:41.928559 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62af3d-d3a8-4dcd-be4a-d53b6a019bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:41.932215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62af3d-d3a8-4dcd-be4a-d53b6a019bac None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:41.932530 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62af3d-d3a8-4dcd-be4a-d53b6a019bac None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:41.932829 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62af3d-d3a8-4dcd-be4a-d53b6a019bac None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:41.933512 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62af3d-d3a8-4dcd-be4a-d53b6a019bac None admin] RBAC: Adding query filter params (type=image) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:41.933850 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62af3d-d3a8-4dcd-be4a-d53b6a019bac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:41.935196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62af3d-d3a8-4dcd-be4a-d53b6a019bac None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:41.945775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 515/1037] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:41.946845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62af3d-d3a8-4dcd-be4a-d53b6a019bac None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:41.973612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573afacd-4ce2-45a0-a6b4-ff238adeb50d None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:41.977426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:42.012500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7e77b-b742-45d5-8950-f240becdd226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:42.013152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7e77b-b742-45d5-8950-f240becdd226 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:42.013449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7e77b-b742-45d5-8950-f240becdd226 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:42.013721 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7e77b-b742-45d5-8950-f240becdd226 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:42.060439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62af3d-d3a8-4dcd-be4a-d53b6a019bac None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:42.097158 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e6d1a-7729-41fc-9ad8-959913dd9e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:42.097799 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e6d1a-7729-41fc-9ad8-959913dd9e11 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:42.098080 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e6d1a-7729-41fc-9ad8-959913dd9e11 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:42.098357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e6d1a-7729-41fc-9ad8-959913dd9e11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:42.099769 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e6d1a-7729-41fc-9ad8-959913dd9e11 None admin] RBAC: Adding query filter params (interface=public, service_id=b1e4ede3ad7145459536a884f5c3e9af, region_id=RegionOne) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:42.100218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e6d1a-7729-41fc-9ad8-959913dd9e11 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:42.101688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e6d1a-7729-41fc-9ad8-959913dd9e11 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:42.105782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7e77b-b742-45d5-8950-f240becdd226 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:42.107649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 516/1038] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Aug 3 18:38:41 2022] GET /identity/v3/endpoints?service_id=b1e4ede3ad7145459536a884f5c3e9af&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:42.108370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7e77b-b742-45d5-8950-f240becdd226 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:42.113040 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e6d1a-7729-41fc-9ad8-959913dd9e11 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:42.164599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:42.209599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:42.223117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 523/1039] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:38:41 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 18:38:42.223314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e6d1a-7729-41fc-9ad8-959913dd9e11 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:42.225301 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7e77b-b742-45d5-8950-f240becdd226 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:42.260129 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799152f8-77bc-4213-a9a0-98ea3a450e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:42.261074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799152f8-77bc-4213-a9a0-98ea3a450e8d None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:42.261513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799152f8-77bc-4213-a9a0-98ea3a450e8d None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:42.261953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799152f8-77bc-4213-a9a0-98ea3a450e8d None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:42.262606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799152f8-77bc-4213-a9a0-98ea3a450e8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:42.263960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799152f8-77bc-4213-a9a0-98ea3a450e8d None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:42.270732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 517/1040] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Aug 3 18:38:42 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:38:42.339433 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7e77b-b742-45d5-8950-f240becdd226 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:42.381645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7630b-0fba-4eb4-8c26-4a5d880e08f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:42.382512 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7630b-0fba-4eb4-8c26-4a5d880e08f8 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:42.383049 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7630b-0fba-4eb4-8c26-4a5d880e08f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:42.383598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7630b-0fba-4eb4-8c26-4a5d880e08f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:42.396770 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7630b-0fba-4eb4-8c26-4a5d880e08f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:42.398986 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7630b-0fba-4eb4-8c26-4a5d880e08f8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:42.417409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-49a7630b-0fba-4eb4-8c26-4a5d880e08f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:42.417409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:42.417409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:42.417409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:42.417409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:42.417409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:42.417409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:42.417409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:42.417409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:42.417409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:42.417409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:42.417409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:42.417409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:42.417409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:42.417409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:42.418541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:42.419663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:42.419663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:42.419663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:42.419663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:42.420118 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 524/1041] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:42.425919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799152f8-77bc-4213-a9a0-98ea3a450e8d None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:42.540157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799152f8-77bc-4213-a9a0-98ea3a450e8d None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:42.574832 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d55cd-ca9a-4a7b-abf0-5b5eeeb846b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:42.575537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d55cd-ca9a-4a7b-abf0-5b5eeeb846b3 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:42.575890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d55cd-ca9a-4a7b-abf0-5b5eeeb846b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:42.576237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d55cd-ca9a-4a7b-abf0-5b5eeeb846b3 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:42.577047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d55cd-ca9a-4a7b-abf0-5b5eeeb846b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:42.577422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d55cd-ca9a-4a7b-abf0-5b5eeeb846b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:42.578849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d55cd-ca9a-4a7b-abf0-5b5eeeb846b3 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:42.591361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 518/1042] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:42.596741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7630b-0fba-4eb4-8c26-4a5d880e08f8 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:42.600582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:42.704365 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7630b-0fba-4eb4-8c26-4a5d880e08f8 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:42.739436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e5e1b-f808-4d17-892f-4e01673909a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:42.740145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e5e1b-f808-4d17-892f-4e01673909a0 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:42.740551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e5e1b-f808-4d17-892f-4e01673909a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:42.740833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e5e1b-f808-4d17-892f-4e01673909a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:42.748896 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e5e1b-f808-4d17-892f-4e01673909a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:42.750362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e5e1b-f808-4d17-892f-4e01673909a0 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:42.763620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-216e5e1b-f808-4d17-892f-4e01673909a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:42.763620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:42.763620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:42.763620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:42.763620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:42.763620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:42.763620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:42.763620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:42.763620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:42.763620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:42.763620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:42.763620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:42.763620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:42.763620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:42.763620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:42.764853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:42.766113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:42.766113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:42.766113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:42.766113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:42.766113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:42.766113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:42.766113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:42.766113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:42.768693 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 525/1043] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:42.770603 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d55cd-ca9a-4a7b-abf0-5b5eeeb846b3 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:42.789342 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:42.879156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d55cd-ca9a-4a7b-abf0-5b5eeeb846b3 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:42.921057 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647116d-9fd8-47be-b695-15fc5306861a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:42.921686 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647116d-9fd8-47be-b695-15fc5306861a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:42.921971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647116d-9fd8-47be-b695-15fc5306861a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:42.922250 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647116d-9fd8-47be-b695-15fc5306861a None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:42.923018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647116d-9fd8-47be-b695-15fc5306861a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:42.923328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647116d-9fd8-47be-b695-15fc5306861a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:42.924830 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647116d-9fd8-47be-b695-15fc5306861a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:42.944369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 519/1044] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:38:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:42.946754 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e5e1b-f808-4d17-892f-4e01673909a0 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:43.012918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:43.060151 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e5e1b-f808-4d17-892f-4e01673909a0 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:43.094464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8cf13-4c48-4bfd-810c-27e53f359038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:43.095085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8cf13-4c48-4bfd-810c-27e53f359038 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:43.095368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8cf13-4c48-4bfd-810c-27e53f359038 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:43.095645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8cf13-4c48-4bfd-810c-27e53f359038 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:43.096476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8cf13-4c48-4bfd-810c-27e53f359038 None admin] RBAC: Adding query filter params (role.id=6dcc297751a745d38cb3a184f49cfd60, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=850a50cfe4394febbafb0908b08888b2) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:43.096793 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8cf13-4c48-4bfd-810c-27e53f359038 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:43.098224 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8cf13-4c48-4bfd-810c-27e53f359038 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:43.104607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 526/1045] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:38:42 2022] GET /identity/v3/role_assignments?user.id=850a50cfe4394febbafb0908b08888b2&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=6dcc297751a745d38cb3a184f49cfd60 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:43.303663 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79afbe3-cab0-49fb-8c7a-ede349f53d80 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:43.304011 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79afbe3-cab0-49fb-8c7a-ede349f53d80 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:43.304189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79afbe3-cab0-49fb-8c7a-ede349f53d80 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:43.305721 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 520/1046] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:43.310824 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b0b74-79f7-48cf-a01e-499b539fa009 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:43.311117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b0b74-79f7-48cf-a01e-499b539fa009 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:43.311397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b0b74-79f7-48cf-a01e-499b539fa009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:43.330395 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d02b0b74-79f7-48cf-a01e-499b539fa009 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:43.331582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d02b0b74-79f7-48cf-a01e-499b539fa009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:43.402290 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 527/1047] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:43.425020 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79afbe3-cab0-49fb-8c7a-ede349f53d80 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:43.516946 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:43.546148 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79afbe3-cab0-49fb-8c7a-ede349f53d80 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:43.580529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c1a28-3345-4467-b6ec-e4bc320a2105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:43.581244 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c1a28-3345-4467-b6ec-e4bc320a2105 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:43.581629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c1a28-3345-4467-b6ec-e4bc320a2105 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:43.582006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c1a28-3345-4467-b6ec-e4bc320a2105 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:43.593204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c1a28-3345-4467-b6ec-e4bc320a2105 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-2d3c1a28-3345-4467-b6ec-e4bc320a2105 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:43.598396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:43.600041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:43.601151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:43.601151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:43.601151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:43.601151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:43.601151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:43.601151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:43.601151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:43.601151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:43.601151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:43.601151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:43.601151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:43.601151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:43.607188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 521/1048] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:38:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:43.608479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b0b74-79f7-48cf-a01e-499b539fa009 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:43.667283 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:43.714114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b0b74-79f7-48cf-a01e-499b539fa009 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:43.750144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973a98d-25ed-4e6f-b98b-764e92bfcd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:43.750772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973a98d-25ed-4e6f-b98b-764e92bfcd6a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:43.751053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973a98d-25ed-4e6f-b98b-764e92bfcd6a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:43.751328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973a98d-25ed-4e6f-b98b-764e92bfcd6a None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:43.752048 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7973a98d-25ed-4e6f-b98b-764e92bfcd6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:43.752633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7973a98d-25ed-4e6f-b98b-764e92bfcd6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:43.754007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7973a98d-25ed-4e6f-b98b-764e92bfcd6a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:43.768007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 528/1049] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:38:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:43.769584 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c1a28-3345-4467-b6ec-e4bc320a2105 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:43.879506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c1a28-3345-4467-b6ec-e4bc320a2105 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:43.918387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7b2c5-e435-4c8d-973f-5d13b71d518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:43.919049 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7b2c5-e435-4c8d-973f-5d13b71d518f None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:43.919339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7b2c5-e435-4c8d-973f-5d13b71d518f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:43.919746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7b2c5-e435-4c8d-973f-5d13b71d518f None admin] PATH_INFO: `/v3/users/902a7684b2fb47c18238631b71b8f62a` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:43.921962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:43.940342 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7b2c5-e435-4c8d-973f-5d13b71d518f None admin] RBAC: Authorizing `identity:get_user(user_id=902a7684b2fb47c18238631b71b8f62a)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:43.942088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7b2c5-e435-4c8d-973f-5d13b71d518f None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:43.944067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 522/1050] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:38:43 2022] GET /identity/v3/users/902a7684b2fb47c18238631b71b8f62a => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:43.950044 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973a98d-25ed-4e6f-b98b-764e92bfcd6a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:44.055263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973a98d-25ed-4e6f-b98b-764e92bfcd6a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:44.072765 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:44.092425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a2a99-3847-498f-9f84-f17ebcaa90d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:44.093049 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2a99-3847-498f-9f84-f17ebcaa90d7 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:44.093332 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2a99-3847-498f-9f84-f17ebcaa90d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:44.093601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2a99-3847-498f-9f84-f17ebcaa90d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:44.164796 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a2a99-3847-498f-9f84-f17ebcaa90d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:44.166165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a2a99-3847-498f-9f84-f17ebcaa90d7 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:44.270380 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 529/1051] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:38:43 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:38:44.276185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7b2c5-e435-4c8d-973f-5d13b71d518f None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:44.346934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69361f-9bbc-410b-aa6b-84e0fdf1d417 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:44.347254 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69361f-9bbc-410b-aa6b-84e0fdf1d417 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:44.347498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69361f-9bbc-410b-aa6b-84e0fdf1d417 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:44.349018 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 530/1052] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:38:44.353757 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c588a-cd98-43c9-9e02-2719d6d8dd1e None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:44.354047 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c588a-cd98-43c9-9e02-2719d6d8dd1e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:44.354325 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c588a-cd98-43c9-9e02-2719d6d8dd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:44.373194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-4c4c588a-cd98-43c9-9e02-2719d6d8dd1e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:44.374372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4c4c588a-cd98-43c9-9e02-2719d6d8dd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:44.383953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7b2c5-e435-4c8d-973f-5d13b71d518f None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:44.389082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:44.424132 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0517f3-d59f-4d1b-997c-8355fbd5c45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:44.424722 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0517f3-d59f-4d1b-997c-8355fbd5c45b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:44.425010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0517f3-d59f-4d1b-997c-8355fbd5c45b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:44.425285 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0517f3-d59f-4d1b-997c-8355fbd5c45b None admin] PATH_INFO: `/v3/projects/ab79d54bb93d433e90bd44b897dd4c57` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:44.426437 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0517f3-d59f-4d1b-997c-8355fbd5c45b None admin] RBAC: Authorizing `identity:get_project(project_id=ab79d54bb93d433e90bd44b897dd4c57)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:44.428500 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0517f3-d59f-4d1b-997c-8355fbd5c45b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:44.430181 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 523/1053] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:38:44 2022] GET /identity/v3/projects/ab79d54bb93d433e90bd44b897dd4c57 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:44.436027 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0517f3-d59f-4d1b-997c-8355fbd5c45b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:44.456304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 531/1054] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:44.478150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c588a-cd98-43c9-9e02-2719d6d8dd1e None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:44.502511 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:44.546870 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0517f3-d59f-4d1b-997c-8355fbd5c45b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:44.583373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc47b88-3929-4897-a07c-e2cf2ce14d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:44.584208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc47b88-3929-4897-a07c-e2cf2ce14d94 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:44.584649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc47b88-3929-4897-a07c-e2cf2ce14d94 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:44.584925 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc47b88-3929-4897-a07c-e2cf2ce14d94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:44.585724 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc47b88-3929-4897-a07c-e2cf2ce14d94 None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=ab79d54bb93d433e90bd44b897dd4c57, user.id=902a7684b2fb47c18238631b71b8f62a) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:44.586027 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc47b88-3929-4897-a07c-e2cf2ce14d94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:44.587445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc47b88-3929-4897-a07c-e2cf2ce14d94 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:44.593091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 524/1055] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:38:44 2022] GET /identity/v3/role_assignments?user.id=902a7684b2fb47c18238631b71b8f62a&scope.project.id=ab79d54bb93d433e90bd44b897dd4c57&role.id=19557e941d6f475ba92cf862846f9872 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:44.613997 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c588a-cd98-43c9-9e02-2719d6d8dd1e None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:44.655897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f67e2a-f9de-401a-9e87-184934ccf354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:44.658938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f67e2a-f9de-401a-9e87-184934ccf354 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:44.658938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f67e2a-f9de-401a-9e87-184934ccf354 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:44.658938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f67e2a-f9de-401a-9e87-184934ccf354 None admin] PATH_INFO: `/v3/services/image` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:44.658938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f67e2a-f9de-401a-9e87-184934ccf354 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:44.659256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f67e2a-f9de-401a-9e87-184934ccf354 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-71f67e2a-f9de-401a-9e87-184934ccf354 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:44.670244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:44.671303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:44.672159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:44.672159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:44.672159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:44.672159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:44.672159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:44.672159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:44.672159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:44.672159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:44.672159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:44.672159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:44.672159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:38:44.672159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:38:44.672159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 18:38:44.672159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:44.676754 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 532/1056] 10.222.0.69 () {56 vars in 1105 bytes} [Wed Aug 3 18:38:44 2022] GET /identity/v3/services/image => generated 86 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:44.678830 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc47b88-3929-4897-a07c-e2cf2ce14d94 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:44.800261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc47b88-3929-4897-a07c-e2cf2ce14d94 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:44.827213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:44.839827 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009b624-9738-4aa1-a220-d100761cfdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:44.840541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009b624-9738-4aa1-a220-d100761cfdca None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:44.840821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009b624-9738-4aa1-a220-d100761cfdca None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:44.841092 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009b624-9738-4aa1-a220-d100761cfdca None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:44.842051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009b624-9738-4aa1-a220-d100761cfdca None admin] RBAC: Adding query filter params (name=image) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:44.842355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009b624-9738-4aa1-a220-d100761cfdca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:44.843702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009b624-9738-4aa1-a220-d100761cfdca None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:44.852379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 525/1057] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:44.855195 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f67e2a-f9de-401a-9e87-184934ccf354 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:44.876103 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:44.978158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f67e2a-f9de-401a-9e87-184934ccf354 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:45.013810 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b43cf-5aa8-4f60-afb2-9cf89b9cfa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:45.014436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b43cf-5aa8-4f60-afb2-9cf89b9cfa2d None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:45.014718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b43cf-5aa8-4f60-afb2-9cf89b9cfa2d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:45.014994 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b43cf-5aa8-4f60-afb2-9cf89b9cfa2d None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:45.015676 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b43cf-5aa8-4f60-afb2-9cf89b9cfa2d None admin] RBAC: Adding query filter params (type=image) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:45.015981 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b43cf-5aa8-4f60-afb2-9cf89b9cfa2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:45.017391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b43cf-5aa8-4f60-afb2-9cf89b9cfa2d None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:45.028044 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 533/1058] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:45.029058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009b624-9738-4aa1-a220-d100761cfdca None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:45.129202 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29225f26-f271-412c-884d-b26be7d85c13 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:45.129860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29225f26-f271-412c-884d-b26be7d85c13 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:45.130233 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29225f26-f271-412c-884d-b26be7d85c13 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:45.132982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 534/1059] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:38:45.140339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dbcf1-1ad9-4183-b77a-9537ef0d07bf None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:45.140339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dbcf1-1ad9-4183-b77a-9537ef0d07bf None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:45.140339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dbcf1-1ad9-4183-b77a-9537ef0d07bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:45.140661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009b624-9738-4aa1-a220-d100761cfdca None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:45.158370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-799dbcf1-1ad9-4183-b77a-9537ef0d07bf None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:45.159572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-799dbcf1-1ad9-4183-b77a-9537ef0d07bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:45.180742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eab99d-d608-4de6-8fba-7554fe9bec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:45.181375 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eab99d-d608-4de6-8fba-7554fe9bec44 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:45.181657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eab99d-d608-4de6-8fba-7554fe9bec44 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:45.181931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eab99d-d608-4de6-8fba-7554fe9bec44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:45.182648 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eab99d-d608-4de6-8fba-7554fe9bec44 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:45.184130 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eab99d-d608-4de6-8fba-7554fe9bec44 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:45.216327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 526/1060] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:38:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:38:45.251690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 535/1061] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:45.281094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eab99d-d608-4de6-8fba-7554fe9bec44 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:45.334163 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:45.398356 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eab99d-d608-4de6-8fba-7554fe9bec44 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:45.434726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f1b54-64ab-40fd-bcd3-f554be7891c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:45.435287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f1b54-64ab-40fd-bcd3-f554be7891c5 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:45.435568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f1b54-64ab-40fd-bcd3-f554be7891c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:45.435839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f1b54-64ab-40fd-bcd3-f554be7891c5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:45.445953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f1b54-64ab-40fd-bcd3-f554be7891c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-335f1b54-64ab-40fd-bcd3-f554be7891c5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:45.451187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:45.452349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:45.453227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:45.453227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:45.453227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:45.453227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:45.453227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:45.453227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:45.453227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:45.453227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:45.453227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:45.453227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:45.453227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 18:38:45.453227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:45.453227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 527/1062] 10.222.0.69 () {56 vars in 1120 bytes} [Wed Aug 3 18:38:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:45.458660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dbcf1-1ad9-4183-b77a-9537ef0d07bf None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:45.461090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:45.574006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dbcf1-1ad9-4183-b77a-9537ef0d07bf None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:45.610890 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715a9b6-4cda-41df-a65f-972cc94ef0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:45.611549 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715a9b6-4cda-41df-a65f-972cc94ef0c2 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:45.611822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715a9b6-4cda-41df-a65f-972cc94ef0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:45.612097 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715a9b6-4cda-41df-a65f-972cc94ef0c2 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:45.612876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715a9b6-4cda-41df-a65f-972cc94ef0c2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:45.613188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715a9b6-4cda-41df-a65f-972cc94ef0c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:45.614662 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715a9b6-4cda-41df-a65f-972cc94ef0c2 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:45.624870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 536/1063] 10.222.0.69 () {56 vars in 1115 bytes} [Wed Aug 3 18:38:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:45.630832 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f1b54-64ab-40fd-bcd3-f554be7891c5 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:45.739107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f1b54-64ab-40fd-bcd3-f554be7891c5 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:45.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:45.779245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed61c7-dffe-4413-b0b2-5534a4664b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:45.779890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed61c7-dffe-4413-b0b2-5534a4664b5e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:45.780172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed61c7-dffe-4413-b0b2-5534a4664b5e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:45.780473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed61c7-dffe-4413-b0b2-5534a4664b5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:45.792243 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed61c7-dffe-4413-b0b2-5534a4664b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:45.793725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed61c7-dffe-4413-b0b2-5534a4664b5e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:45.809145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-cbed61c7-dffe-4413-b0b2-5534a4664b5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:45.809145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:45.809145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:45.809145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:45.809145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:45.809145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:45.809145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:45.809145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:45.809145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:45.809145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:45.809145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:45.809145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:45.809145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:45.809145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:45.809145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:45.810077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:45.810996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:45.810996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:45.810996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:45.810996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:45.810996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 528/1064] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:45.820164 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715a9b6-4cda-41df-a65f-972cc94ef0c2 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:45.884963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:45.927533 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715a9b6-4cda-41df-a65f-972cc94ef0c2 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:45.962712 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f46723-323a-4292-afac-51a87c833df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:45.963338 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f46723-323a-4292-afac-51a87c833df3 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:45.963620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f46723-323a-4292-afac-51a87c833df3 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:45.963894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f46723-323a-4292-afac-51a87c833df3 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:45.964629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f46723-323a-4292-afac-51a87c833df3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:45.964944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f46723-323a-4292-afac-51a87c833df3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:45.966313 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f46723-323a-4292-afac-51a87c833df3 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:45.983374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 537/1065] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:45.984636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed61c7-dffe-4413-b0b2-5534a4664b5e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:46.089716 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed61c7-dffe-4413-b0b2-5534a4664b5e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:46.127885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67308be9-b736-47a8-85ca-f528342e7c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:46.128557 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67308be9-b736-47a8-85ca-f528342e7c08 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:46.128845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67308be9-b736-47a8-85ca-f528342e7c08 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:46.129120 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67308be9-b736-47a8-85ca-f528342e7c08 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:46.136954 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67308be9-b736-47a8-85ca-f528342e7c08 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:46.138402 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67308be9-b736-47a8-85ca-f528342e7c08 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-67308be9-b736-47a8-85ca-f528342e7c08 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:38:46.149746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:46.150745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:46.151739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:46.151739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:46.151739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:38:46.151739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:38:46.151739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:38:46.151739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:38:46.151739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:38:46.151739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:38:46.151739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:38:46.151739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:46.151739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 529/1066] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Aug 3 18:38:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:46.158459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f46723-323a-4292-afac-51a87c833df3 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:46.261998 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f46723-323a-4292-afac-51a87c833df3 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:46.286705 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:46.298478 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c7b10-7b4f-4315-9d6f-7627c59a5b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:46.299375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c7b10-7b4f-4315-9d6f-7627c59a5b3c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:46.299853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c7b10-7b4f-4315-9d6f-7627c59a5b3c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:46.300333 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c7b10-7b4f-4315-9d6f-7627c59a5b3c None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:46.301321 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c7b10-7b4f-4315-9d6f-7627c59a5b3c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:46.301815 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c7b10-7b4f-4315-9d6f-7627c59a5b3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:46.303496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c7b10-7b4f-4315-9d6f-7627c59a5b3c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:46.322400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 538/1067] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Aug 3 18:38:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:46.328221 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67308be9-b736-47a8-85ca-f528342e7c08 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:46.348664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:46.434932 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67308be9-b736-47a8-85ca-f528342e7c08 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:46.472056 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85c02f-0e1c-4757-974c-602a3f1c997e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:46.473149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85c02f-0e1c-4757-974c-602a3f1c997e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:46.473635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85c02f-0e1c-4757-974c-602a3f1c997e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:46.474237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85c02f-0e1c-4757-974c-602a3f1c997e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:46.485590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85c02f-0e1c-4757-974c-602a3f1c997e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:46.487485 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85c02f-0e1c-4757-974c-602a3f1c997e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:46.504886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-1c85c02f-0e1c-4757-974c-602a3f1c997e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:46.504886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:46.504886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:46.504886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:46.504886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:46.504886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:46.504886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:46.504886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:46.504886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:46.504886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:46.504886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:46.504886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:46.504886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:46.504886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:46.504886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:46.506676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:46.508193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:46.508193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:46.508193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:46.508193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:46.508193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 530/1068] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:46.512541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c7b10-7b4f-4315-9d6f-7627c59a5b3c None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:46.614933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c7b10-7b4f-4315-9d6f-7627c59a5b3c None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:46.650525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee23f20-2e80-42a4-9957-a9fa72af849a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:46.651156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee23f20-2e80-42a4-9957-a9fa72af849a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:46.651441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee23f20-2e80-42a4-9957-a9fa72af849a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:46.651718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee23f20-2e80-42a4-9957-a9fa72af849a None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:46.652444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee23f20-2e80-42a4-9957-a9fa72af849a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:46.652753 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee23f20-2e80-42a4-9957-a9fa72af849a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:46.654126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee23f20-2e80-42a4-9957-a9fa72af849a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:46.671706 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 539/1069] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:46.674441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85c02f-0e1c-4757-974c-602a3f1c997e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:46.689571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c704e1-73a2-47c2-b44f-407e16dea847 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:46.689861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c704e1-73a2-47c2-b44f-407e16dea847 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:46.690128 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c704e1-73a2-47c2-b44f-407e16dea847 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:46.691621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 540/1070] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:38:46.697353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d2a9f-6c22-429a-991d-7e92fa19b21e None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:46.697516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d2a9f-6c22-429a-991d-7e92fa19b21e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:46.697935 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d2a9f-6c22-429a-991d-7e92fa19b21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:46.717243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-1d8d2a9f-6c22-429a-991d-7e92fa19b21e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:46.718428 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1d8d2a9f-6c22-429a-991d-7e92fa19b21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:46.748078 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:46.759548 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:46.791477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85c02f-0e1c-4757-974c-602a3f1c997e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:46.795786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 541/1071] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:46.821214 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d2a9f-6c22-429a-991d-7e92fa19b21e None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:46.828584 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776fb58-4a47-4ab7-93f4-d82f758f0c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:46.829232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776fb58-4a47-4ab7-93f4-d82f758f0c8e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:46.829518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776fb58-4a47-4ab7-93f4-d82f758f0c8e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:46.829793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776fb58-4a47-4ab7-93f4-d82f758f0c8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:46.837279 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776fb58-4a47-4ab7-93f4-d82f758f0c8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:46.838732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776fb58-4a47-4ab7-93f4-d82f758f0c8e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:46.851088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-0776fb58-4a47-4ab7-93f4-d82f758f0c8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:46.851088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:46.851088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:46.851088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:46.851088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:46.851088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:46.851088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:46.851088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:46.851088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:46.851088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:46.851088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:46.851088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:46.851088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:46.851088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:46.851088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:46.852010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:46.852988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:46.852988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:46.852988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:46.852988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:46.852988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:46.852988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:46.852988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:46.852988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:46.852988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 531/1072] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:46.857913 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776fb58-4a47-4ab7-93f4-d82f758f0c8e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:46.941789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d2a9f-6c22-429a-991d-7e92fa19b21e None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:46.967783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776fb58-4a47-4ab7-93f4-d82f758f0c8e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:46.984325 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee98767-818d-4a7a-a19c-082dd17fe030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:46.985140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee98767-818d-4a7a-a19c-082dd17fe030 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:46.985669 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee98767-818d-4a7a-a19c-082dd17fe030 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:46.985968 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee98767-818d-4a7a-a19c-082dd17fe030 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:46.995997 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:47.001516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee98767-818d-4a7a-a19c-082dd17fe030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:47.004436 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7225c8ab-1a81-4945-8a68-ba86f4599c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:47.005172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225c8ab-1a81-4945-8a68-ba86f4599c87 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:47.005482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225c8ab-1a81-4945-8a68-ba86f4599c87 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:47.005773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225c8ab-1a81-4945-8a68-ba86f4599c87 None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:47.006674 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7225c8ab-1a81-4945-8a68-ba86f4599c87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:47.006984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7225c8ab-1a81-4945-8a68-ba86f4599c87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:47.008432 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7225c8ab-1a81-4945-8a68-ba86f4599c87 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-aee98767-818d-4a7a-a19c-082dd17fe030 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:47.008641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:47.009734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:47.010784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:47.010784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:47.010784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:47.010784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:47.010784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:47.010784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:47.010784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:47.010784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:47.010784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:47.010784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:47.010784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:47.010784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:47.018904 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 542/1073] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:38:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:47.019574 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee98767-818d-4a7a-a19c-082dd17fe030 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:47.024094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 532/1074] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:38:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:47.029881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7225c8ab-1a81-4945-8a68-ba86f4599c87 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:47.053564 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:47.136182 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7225c8ab-1a81-4945-8a68-ba86f4599c87 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:47.140562 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee98767-818d-4a7a-a19c-082dd17fe030 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:47.175848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46f590-f2b5-40b9-9f98-0ca3ac91a471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:47.176558 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46f590-f2b5-40b9-9f98-0ca3ac91a471 None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:47.176865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46f590-f2b5-40b9-9f98-0ca3ac91a471 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:47.177843 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46f590-f2b5-40b9-9f98-0ca3ac91a471 None admin] PATH_INFO: `/v3/projects/59f00389bb404b4392d98e22e045c84f/users/850a50cfe4394febbafb0908b08888b2/roles/6dcc297751a745d38cb3a184f49cfd60` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:47.178465 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d25882-42cb-4346-8ce1-f5f270a4cebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:47.179091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d25882-42cb-4346-8ce1-f5f270a4cebe None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:47.179368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d25882-42cb-4346-8ce1-f5f270a4cebe None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:47.179637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d25882-42cb-4346-8ce1-f5f270a4cebe None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:47.180323 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46f590-f2b5-40b9-9f98-0ca3ac91a471 None admin] RBAC: Authorizing `identity:create_grant(project_id=59f00389bb404b4392d98e22e045c84f, user_id=850a50cfe4394febbafb0908b08888b2, role_id=6dcc297751a745d38cb3a184f49cfd60)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:47.180438 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d25882-42cb-4346-8ce1-f5f270a4cebe None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:47.180669 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d25882-42cb-4346-8ce1-f5f270a4cebe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:47.181770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46f590-f2b5-40b9-9f98-0ca3ac91a471 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:47.182022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d25882-42cb-4346-8ce1-f5f270a4cebe None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:47.192413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 543/1075] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:38:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:47.197641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 533/1076] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:38:47 2022] PUT /identity/v3/projects/59f00389bb404b4392d98e22e045c84f/users/850a50cfe4394febbafb0908b08888b2/roles/6dcc297751a745d38cb3a184f49cfd60 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:38:47.198744 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d25882-42cb-4346-8ce1-f5f270a4cebe None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:47.292946 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c963563-4f13-443e-b08c-4503126ae22d None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:47.293150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c963563-4f13-443e-b08c-4503126ae22d None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:47.293379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c963563-4f13-443e-b08c-4503126ae22d None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:47.294745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 534/1077] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:38:47.300192 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0c8c5-d256-4c20-8590-1486a85f8a46 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:47.300502 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0c8c5-d256-4c20-8590-1486a85f8a46 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:47.300784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0c8c5-d256-4c20-8590-1486a85f8a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:47.316315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:47.321873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-84b0c8c5-d256-4c20-8590-1486a85f8a46 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:47.323420 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-84b0c8c5-d256-4c20-8590-1486a85f8a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:47.327295 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d25882-42cb-4346-8ce1-f5f270a4cebe None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:47.369769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e42e5-d9bc-4f18-8d26-09ea2862af7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:47.370434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e42e5-d9bc-4f18-8d26-09ea2862af7e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:47.370724 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e42e5-d9bc-4f18-8d26-09ea2862af7e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:47.370999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e42e5-d9bc-4f18-8d26-09ea2862af7e None admin] PATH_INFO: `/v3/users/902a7684b2fb47c18238631b71b8f62a` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:47.372488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e42e5-d9bc-4f18-8d26-09ea2862af7e None admin] RBAC: Authorizing `identity:get_user(user_id=902a7684b2fb47c18238631b71b8f62a)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:47.373669 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e42e5-d9bc-4f18-8d26-09ea2862af7e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:47.380315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 544/1078] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:38:47 2022] GET /identity/v3/users/902a7684b2fb47c18238631b71b8f62a => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:47.381041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:47.381891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e42e5-d9bc-4f18-8d26-09ea2862af7e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:47.409984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 535/1079] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:47.433211 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0c8c5-d256-4c20-8590-1486a85f8a46 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:47.495930 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e42e5-d9bc-4f18-8d26-09ea2862af7e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:47.533997 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fb3d6-ceb4-452d-9cfa-24f0cfac24d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:47.535084 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fb3d6-ceb4-452d-9cfa-24f0cfac24d0 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:47.535366 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fb3d6-ceb4-452d-9cfa-24f0cfac24d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:47.535806 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fb3d6-ceb4-452d-9cfa-24f0cfac24d0 None admin] PATH_INFO: `/v3/projects/ab79d54bb93d433e90bd44b897dd4c57` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:47.537286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3fb3d6-ceb4-452d-9cfa-24f0cfac24d0 None admin] RBAC: Authorizing `identity:get_project(project_id=ab79d54bb93d433e90bd44b897dd4c57)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:47.538792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3fb3d6-ceb4-452d-9cfa-24f0cfac24d0 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:47.540402 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 545/1080] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:38:47 2022] GET /identity/v3/projects/ab79d54bb93d433e90bd44b897dd4c57 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:47.546151 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fb3d6-ceb4-452d-9cfa-24f0cfac24d0 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:47.551605 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0c8c5-d256-4c20-8590-1486a85f8a46 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:47.580617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:47.588614 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbfffc-8e8d-4c10-ac01-9237f9d4c015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:47.589250 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbfffc-8e8d-4c10-ac01-9237f9d4c015 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:47.589535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbfffc-8e8d-4c10-ac01-9237f9d4c015 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:47.589812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbfffc-8e8d-4c10-ac01-9237f9d4c015 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:47.638586 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:47.659697 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fb3d6-ceb4-452d-9cfa-24f0cfac24d0 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:47.669199 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbfffc-8e8d-4c10-ac01-9237f9d4c015 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:47.670725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbfffc-8e8d-4c10-ac01-9237f9d4c015 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:47.703444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70336bd7-0e8d-409d-886b-f496a4507003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:47.704132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70336bd7-0e8d-409d-886b-f496a4507003 None admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:47.704464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70336bd7-0e8d-409d-886b-f496a4507003 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:47.704766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70336bd7-0e8d-409d-886b-f496a4507003 None admin] PATH_INFO: `/v3/projects/ab79d54bb93d433e90bd44b897dd4c57/users/902a7684b2fb47c18238631b71b8f62a/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:47.706698 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70336bd7-0e8d-409d-886b-f496a4507003 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab79d54bb93d433e90bd44b897dd4c57, user_id=902a7684b2fb47c18238631b71b8f62a, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:47.708196 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70336bd7-0e8d-409d-886b-f496a4507003 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:47.722774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 546/1081] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:38:47 2022] PUT /identity/v3/projects/ab79d54bb93d433e90bd44b897dd4c57/users/902a7684b2fb47c18238631b71b8f62a/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:38:47.801960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 536/1082] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:38:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 18:38:47.803268 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70336bd7-0e8d-409d-886b-f496a4507003 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:47.920022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:47.936097 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70336bd7-0e8d-409d-886b-f496a4507003 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:47.973602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab566ff3-9685-4e30-af0b-eb2cb81da8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:47.974414 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab566ff3-9685-4e30-af0b-eb2cb81da8d2 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:47.974634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab566ff3-9685-4e30-af0b-eb2cb81da8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:47.974962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab566ff3-9685-4e30-af0b-eb2cb81da8d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:47.986857 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab566ff3-9685-4e30-af0b-eb2cb81da8d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:47.988380 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab566ff3-9685-4e30-af0b-eb2cb81da8d2 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:48.004909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-ab566ff3-9685-4e30-af0b-eb2cb81da8d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:48.004909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:48.004909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:48.004909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:48.004909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:48.004909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:48.004909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:48.004909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:48.004909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:48.004909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:48.004909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:48.004909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:48.004909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:48.004909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:48.004909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:48.006509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:48.007932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:48.007932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:48.007932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:48.007932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:48.007932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 547/1083] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:48.013243 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbfffc-8e8d-4c10-ac01-9237f9d4c015 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:48.101206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:48.127029 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbfffc-8e8d-4c10-ac01-9237f9d4c015 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:48.184507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc501e-9c10-4b87-993c-bfbee7542c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:48.185443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc501e-9c10-4b87-993c-bfbee7542c82 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:48.185773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc501e-9c10-4b87-993c-bfbee7542c82 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:48.186143 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc501e-9c10-4b87-993c-bfbee7542c82 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:48.187149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc501e-9c10-4b87-993c-bfbee7542c82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:48.187581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc501e-9c10-4b87-993c-bfbee7542c82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:48.189726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc501e-9c10-4b87-993c-bfbee7542c82 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:48.210484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 537/1084] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:49.385294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800909d0-5d3f-47f2-8c27-58c72ac53b35 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:49.385862 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800909d0-5d3f-47f2-8c27-58c72ac53b35 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:49.385862 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800909d0-5d3f-47f2-8c27-58c72ac53b35 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:49.387186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 548/1085] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:49.392422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d027a-882f-4cbc-9c8e-e68d2c0fe5d0 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:49.393167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d027a-882f-4cbc-9c8e-e68d2c0fe5d0 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:49.393450 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d027a-882f-4cbc-9c8e-e68d2c0fe5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:49.413661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-350d027a-882f-4cbc-9c8e-e68d2c0fe5d0 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:49.414845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-350d027a-882f-4cbc-9c8e-e68d2c0fe5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:49.482441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 538/1086] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:49.508063 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800909d0-5d3f-47f2-8c27-58c72ac53b35 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:49.611050 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800909d0-5d3f-47f2-8c27-58c72ac53b35 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:49.645510 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f0be3-aa10-46a0-816b-3743ad2be8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:49.646242 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f0be3-aa10-46a0-816b-3743ad2be8df None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:49.646568 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f0be3-aa10-46a0-816b-3743ad2be8df None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:49.646845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f0be3-aa10-46a0-816b-3743ad2be8df None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:49.657046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f0be3-aa10-46a0-816b-3743ad2be8df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-751f0be3-aa10-46a0-816b-3743ad2be8df None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:49.661991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:49.662932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:49.663791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:49.663791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:49.663791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:49.663791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:49.663791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:49.663791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:49.663791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:49.663791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:49.663791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:49.663791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:49.663791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 18:38:49.663791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:49.663791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 549/1087] 10.222.0.69 () {56 vars in 1120 bytes} [Wed Aug 3 18:38:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:49.669351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d027a-882f-4cbc-9c8e-e68d2c0fe5d0 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:49.735567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:49.771360 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d027a-882f-4cbc-9c8e-e68d2c0fe5d0 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:49.802920 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922938f1-15ba-401c-b52e-7bd6d1204877 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:49.803204 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922938f1-15ba-401c-b52e-7bd6d1204877 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:49.803489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922938f1-15ba-401c-b52e-7bd6d1204877 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:49.805056 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 550/1088] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:49.806720 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b3feb-6815-41a5-908a-d8d7b074b7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:49.807369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b3feb-6815-41a5-908a-d8d7b074b7f8 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:49.807703 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b3feb-6815-41a5-908a-d8d7b074b7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:49.807976 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b3feb-6815-41a5-908a-d8d7b074b7f8 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:49.808758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b3feb-6815-41a5-908a-d8d7b074b7f8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:49.809061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b3feb-6815-41a5-908a-d8d7b074b7f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:49.809895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02629ee9-ebce-437a-8fee-d435a860ec4a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:49.810182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02629ee9-ebce-437a-8fee-d435a860ec4a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:49.810425 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b3feb-6815-41a5-908a-d8d7b074b7f8 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:49.810518 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02629ee9-ebce-437a-8fee-d435a860ec4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:49.817239 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:49.823995 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 539/1089] 10.222.0.69 () {56 vars in 1115 bytes} [Wed Aug 3 18:38:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:49.825126 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b3feb-6815-41a5-908a-d8d7b074b7f8 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:49.829792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-02629ee9-ebce-437a-8fee-d435a860ec4a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:49.831009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-02629ee9-ebce-437a-8fee-d435a860ec4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:49.902455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 551/1090] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:49.925673 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02629ee9-ebce-437a-8fee-d435a860ec4a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:49.927090 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b3feb-6815-41a5-908a-d8d7b074b7f8 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:49.956168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:49.963246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c1f4b-1d45-4ded-8bf9-3a61ecacd62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:49.963891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c1f4b-1d45-4ded-8bf9-3a61ecacd62a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:49.964172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c1f4b-1d45-4ded-8bf9-3a61ecacd62a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:49.964475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c1f4b-1d45-4ded-8bf9-3a61ecacd62a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:49.975282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c1f4b-1d45-4ded-8bf9-3a61ecacd62a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:49.976899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c1f4b-1d45-4ded-8bf9-3a61ecacd62a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:49.991093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-c72c1f4b-1d45-4ded-8bf9-3a61ecacd62a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:49.991093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:49.991093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:49.991093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:49.991093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:49.991093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:49.991093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:49.991093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:49.991093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:49.991093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:49.991093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:49.991093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:49.991093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:49.991093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:49.991093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:49.992165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:49.993368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:49.993368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:49.993368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:49.993368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:49.996950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 540/1091] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:49.997802 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c1f4b-1d45-4ded-8bf9-3a61ecacd62a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:50.028701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02629ee9-ebce-437a-8fee-d435a860ec4a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:50.057568 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:50.063238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8457bda-b1a5-41e0-9b6a-b95837b13d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:50.063887 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8457bda-b1a5-41e0-9b6a-b95837b13d75 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:50.064168 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8457bda-b1a5-41e0-9b6a-b95837b13d75 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:50.064545 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8457bda-b1a5-41e0-9b6a-b95837b13d75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:50.075362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8457bda-b1a5-41e0-9b6a-b95837b13d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-f8457bda-b1a5-41e0-9b6a-b95837b13d75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:38:50.080240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:50.081238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:50.082319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:50.082319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:50.082319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:50.082319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:50.082319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:38:50.082319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:38:50.082319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:38:50.082319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:38:50.082319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:38:50.082319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:38:50.082319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:38:50.082319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:50.082319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 552/1092] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:38:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:50.088021 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8457bda-b1a5-41e0-9b6a-b95837b13d75 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:50.098168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c1f4b-1d45-4ded-8bf9-3a61ecacd62a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:50.132555 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75181f8-f05e-4f42-95ac-e25d2f937e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:50.133099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75181f8-f05e-4f42-95ac-e25d2f937e91 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:50.133388 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75181f8-f05e-4f42-95ac-e25d2f937e91 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:50.133662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75181f8-f05e-4f42-95ac-e25d2f937e91 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:50.134355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75181f8-f05e-4f42-95ac-e25d2f937e91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:50.134659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75181f8-f05e-4f42-95ac-e25d2f937e91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:50.136805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75181f8-f05e-4f42-95ac-e25d2f937e91 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:50.149891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 541/1093] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:50.155555 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75181f8-f05e-4f42-95ac-e25d2f937e91 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:50.190901 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8457bda-b1a5-41e0-9b6a-b95837b13d75 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:50.198367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:50.226185 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5913b-11b5-4bbd-9244-a229feca7406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:50.226800 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5913b-11b5-4bbd-9244-a229feca7406 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:50.227088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5913b-11b5-4bbd-9244-a229feca7406 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:50.227365 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5913b-11b5-4bbd-9244-a229feca7406 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:50.228082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5913b-11b5-4bbd-9244-a229feca7406 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:50.228418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5913b-11b5-4bbd-9244-a229feca7406 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:50.229787 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5913b-11b5-4bbd-9244-a229feca7406 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:50.237720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 553/1094] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:38:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:50.244095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5913b-11b5-4bbd-9244-a229feca7406 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:50.258193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75181f8-f05e-4f42-95ac-e25d2f937e91 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:50.293184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3e7df-55be-45dd-be64-d4804fd15b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:50.293828 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3e7df-55be-45dd-be64-d4804fd15b89 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:50.294105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3e7df-55be-45dd-be64-d4804fd15b89 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:50.294378 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3e7df-55be-45dd-be64-d4804fd15b89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:50.301195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3e7df-55be-45dd-be64-d4804fd15b89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:50.302639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:50.302773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3e7df-55be-45dd-be64-d4804fd15b89 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-92e3e7df-55be-45dd-be64-d4804fd15b89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:38:50.315446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:50.316534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:50.317694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:50.317694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:50.317694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:38:50.317694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:38:50.317694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:38:50.317694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:38:50.317694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:38:50.317694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:38:50.317694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 18:38:50.317694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:50.317694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 542/1095] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Aug 3 18:38:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:50.322578 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3e7df-55be-45dd-be64-d4804fd15b89 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:50.351972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5913b-11b5-4bbd-9244-a229feca7406 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:50.386623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde45e62-08ef-451d-9818-2ed0482009b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:50.387286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde45e62-08ef-451d-9818-2ed0482009b9 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:50.387525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde45e62-08ef-451d-9818-2ed0482009b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:50.387796 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde45e62-08ef-451d-9818-2ed0482009b9 None admin] PATH_INFO: `/v3/users/902a7684b2fb47c18238631b71b8f62a` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:50.389032 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde45e62-08ef-451d-9818-2ed0482009b9 None admin] RBAC: Authorizing `identity:get_user(user_id=902a7684b2fb47c18238631b71b8f62a)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:50.390451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde45e62-08ef-451d-9818-2ed0482009b9 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:50.392045 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 554/1096] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:38:50 2022] GET /identity/v3/users/902a7684b2fb47c18238631b71b8f62a => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:50.398420 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde45e62-08ef-451d-9818-2ed0482009b9 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:50.429855 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3e7df-55be-45dd-be64-d4804fd15b89 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:50.431615 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:50.464999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e88a91-5ace-4257-860e-d8cd3626ebbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:50.465611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e88a91-5ace-4257-860e-d8cd3626ebbc None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:50.465892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e88a91-5ace-4257-860e-d8cd3626ebbc None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:50.466165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e88a91-5ace-4257-860e-d8cd3626ebbc None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:50.466957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e88a91-5ace-4257-860e-d8cd3626ebbc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:50.467261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e88a91-5ace-4257-860e-d8cd3626ebbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:50.468782 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e88a91-5ace-4257-860e-d8cd3626ebbc None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:50.486879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 543/1097] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Aug 3 18:38:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:50.493250 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e88a91-5ace-4257-860e-d8cd3626ebbc None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:50.503856 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde45e62-08ef-451d-9818-2ed0482009b9 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:50.528334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:50.540035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b74198-6562-42b7-a24b-87cb8e8ae8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:50.540691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b74198-6562-42b7-a24b-87cb8e8ae8f3 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:50.540975 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b74198-6562-42b7-a24b-87cb8e8ae8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:50.541246 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b74198-6562-42b7-a24b-87cb8e8ae8f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:50.594384 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e88a91-5ace-4257-860e-d8cd3626ebbc None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:50.610105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b74198-6562-42b7-a24b-87cb8e8ae8f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:50.611555 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b74198-6562-42b7-a24b-87cb8e8ae8f3 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:50.628557 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97918485-d99d-48aa-8ba3-ea435be2c854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:50.629177 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97918485-d99d-48aa-8ba3-ea435be2c854 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:50.629459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97918485-d99d-48aa-8ba3-ea435be2c854 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:50.629741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97918485-d99d-48aa-8ba3-ea435be2c854 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:50.659192 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:50.700812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97918485-d99d-48aa-8ba3-ea435be2c854 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:50.702202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97918485-d99d-48aa-8ba3-ea435be2c854 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:50.715104 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 555/1098] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:38:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 18:38:50.721054 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b74198-6562-42b7-a24b-87cb8e8ae8f3 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:50.745101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:50.804426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 544/1099] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:38:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:38:50.810242 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97918485-d99d-48aa-8ba3-ea435be2c854 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:50.825088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b74198-6562-42b7-a24b-87cb8e8ae8f3 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:50.859240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b493a02-1512-459e-b5ed-698235c241ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:50.859879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b493a02-1512-459e-b5ed-698235c241ab None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:50.860164 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b493a02-1512-459e-b5ed-698235c241ab None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:50.860479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b493a02-1512-459e-b5ed-698235c241ab None admin] PATH_INFO: `/v3/projects/ab79d54bb93d433e90bd44b897dd4c57` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:50.861978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b493a02-1512-459e-b5ed-698235c241ab None admin] RBAC: Authorizing `identity:get_project(project_id=ab79d54bb93d433e90bd44b897dd4c57)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:50.863083 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b493a02-1512-459e-b5ed-698235c241ab None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:50.864964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 556/1100] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:38:50 2022] GET /identity/v3/projects/ab79d54bb93d433e90bd44b897dd4c57 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:50.870746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:50.871369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b493a02-1512-459e-b5ed-698235c241ab None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:50.910737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97918485-d99d-48aa-8ba3-ea435be2c854 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:50.947055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381fc3a5-9fb8-4dcf-b989-30de75045450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:50.947698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fc3a5-9fb8-4dcf-b989-30de75045450 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:50.947977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fc3a5-9fb8-4dcf-b989-30de75045450 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:50.948249 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fc3a5-9fb8-4dcf-b989-30de75045450 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:50.958182 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381fc3a5-9fb8-4dcf-b989-30de75045450 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:50.960120 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381fc3a5-9fb8-4dcf-b989-30de75045450 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:50.964941 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:50.975184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-381fc3a5-9fb8-4dcf-b989-30de75045450 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:50.975184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:50.975184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:50.975184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:50.975184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:50.975184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:50.975184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:50.975184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:50.975184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:50.975184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:50.975184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:50.975184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:50.975184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:38:50.975184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:38:50.975184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:50.976125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:50.977177 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:38:50.977177 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:38:50.977177 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:38:50.977177 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:38:50.977439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b493a02-1512-459e-b5ed-698235c241ab None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:50.977534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 545/1101] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:38:50.981935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381fc3a5-9fb8-4dcf-b989-30de75045450 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:51.010929 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb87d98-f623-43dd-9549-dd915178e58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:51.011541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb87d98-f623-43dd-9549-dd915178e58c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:51.011877 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb87d98-f623-43dd-9549-dd915178e58c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:51.012159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb87d98-f623-43dd-9549-dd915178e58c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:51.013003 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb87d98-f623-43dd-9549-dd915178e58c None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=ab79d54bb93d433e90bd44b897dd4c57, user.id=902a7684b2fb47c18238631b71b8f62a) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:51.013305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb87d98-f623-43dd-9549-dd915178e58c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:51.014735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb87d98-f623-43dd-9549-dd915178e58c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:51.020398 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 557/1102] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:38:50 2022] GET /identity/v3/role_assignments?user.id=902a7684b2fb47c18238631b71b8f62a&scope.project.id=ab79d54bb93d433e90bd44b897dd4c57&role.id=19557e941d6f475ba92cf862846f9872 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:51.092060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381fc3a5-9fb8-4dcf-b989-30de75045450 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:51.106625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:51.129600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915e630-7840-4962-9137-678e22cebfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:51.130241 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915e630-7840-4962-9137-678e22cebfc2 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:51.130525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915e630-7840-4962-9137-678e22cebfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:51.130801 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915e630-7840-4962-9137-678e22cebfc2 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:51.131519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915e630-7840-4962-9137-678e22cebfc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:51.131818 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915e630-7840-4962-9137-678e22cebfc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:51.133718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915e630-7840-4962-9137-678e22cebfc2 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:51.148979 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 546/1103] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:51.154969 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb87d98-f623-43dd-9549-dd915178e58c None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:51.263153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb87d98-f623-43dd-9549-dd915178e58c None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:51.297798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8d795-43d7-4959-8386-2afff9815f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:51.298444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8d795-43d7-4959-8386-2afff9815f32 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:51.298726 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8d795-43d7-4959-8386-2afff9815f32 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:51.298998 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8d795-43d7-4959-8386-2afff9815f32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:51.306546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8d795-43d7-4959-8386-2afff9815f32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:51.308000 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8d795-43d7-4959-8386-2afff9815f32 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:51.319311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-99f8d795-43d7-4959-8386-2afff9815f32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:51.319311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:38:51.319311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:38:51.319311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:38:51.319311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:38:51.319311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:38:51.319311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:38:51.319311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:38:51.319311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:38:51.319311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:38:51.319311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:38:51.319311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:38:51.319311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:38:51.319311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:38:51.319311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:38:51.320565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:38:51.321479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:38:51.321479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:38:51.321479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:38:51.321479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:38:51.321479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:38:51.321479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:38:51.321479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:38:51.321479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:38:51.321479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 558/1104] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:38:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:38:51.326237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915e630-7840-4962-9137-678e22cebfc2 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:51.426527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915e630-7840-4962-9137-678e22cebfc2 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:51.468395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067c96c-3e0b-4627-94dc-d397ba71321b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:51.469311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067c96c-3e0b-4627-94dc-d397ba71321b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:51.469736 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067c96c-3e0b-4627-94dc-d397ba71321b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:51.470151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067c96c-3e0b-4627-94dc-d397ba71321b None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:51.471325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067c96c-3e0b-4627-94dc-d397ba71321b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:51.471791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067c96c-3e0b-4627-94dc-d397ba71321b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:51.474593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067c96c-3e0b-4627-94dc-d397ba71321b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:51.489354 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 547/1105] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:38:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:51.496480 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8d795-43d7-4959-8386-2afff9815f32 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:51.518411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:51.606587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8d795-43d7-4959-8386-2afff9815f32 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:51.640074 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc0359-82ce-43f4-8d87-a71589ff3a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:51.640713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc0359-82ce-43f4-8d87-a71589ff3a54 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:51.640996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc0359-82ce-43f4-8d87-a71589ff3a54 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:51.641270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc0359-82ce-43f4-8d87-a71589ff3a54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:51.642065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc0359-82ce-43f4-8d87-a71589ff3a54 None admin] RBAC: Adding query filter params (role.id=6dcc297751a745d38cb3a184f49cfd60, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=850a50cfe4394febbafb0908b08888b2) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:51.642374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc0359-82ce-43f4-8d87-a71589ff3a54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:51.643812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc0359-82ce-43f4-8d87-a71589ff3a54 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:51.650257 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 559/1106] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:38:51 2022] GET /identity/v3/role_assignments?user.id=850a50cfe4394febbafb0908b08888b2&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=6dcc297751a745d38cb3a184f49cfd60 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:38:53.080853 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994502be-2277-4302-92d8-9ad90403b86d None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:53.081379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994502be-2277-4302-92d8-9ad90403b86d None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:53.081472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994502be-2277-4302-92d8-9ad90403b86d None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:53.082751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 548/1107] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:53.088772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da455f-5270-4f2c-852e-42f060b6eb17 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:53.089220 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da455f-5270-4f2c-852e-42f060b6eb17 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:53.089322 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da455f-5270-4f2c-852e-42f060b6eb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:53.107206 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-80da455f-5270-4f2c-852e-42f060b6eb17 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:53.108434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-80da455f-5270-4f2c-852e-42f060b6eb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:53.175276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 560/1108] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:53.201370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994502be-2277-4302-92d8-9ad90403b86d None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:53.233050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:53.302962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994502be-2277-4302-92d8-9ad90403b86d None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:53.335673 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cdcc86-df42-47ac-b4cf-e090a282845c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:53.336316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdcc86-df42-47ac-b4cf-e090a282845c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:53.336616 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdcc86-df42-47ac-b4cf-e090a282845c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:53.336885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdcc86-df42-47ac-b4cf-e090a282845c None admin] PATH_INFO: `/v3/domains/default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:53.338012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cdcc86-df42-47ac-b4cf-e090a282845c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:53.339422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cdcc86-df42-47ac-b4cf-e090a282845c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:53.341018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 549/1109] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:38:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:53.346726 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da455f-5270-4f2c-852e-42f060b6eb17 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:53.357076 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:53.446175 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da455f-5270-4f2c-852e-42f060b6eb17 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:53.479270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16119198-93a1-40a3-a30b-e2034e60b016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:53.479890 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16119198-93a1-40a3-a30b-e2034e60b016 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:53.480175 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16119198-93a1-40a3-a30b-e2034e60b016 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:53.480480 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16119198-93a1-40a3-a30b-e2034e60b016 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:53.481173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16119198-93a1-40a3-a30b-e2034e60b016 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:53.482609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16119198-93a1-40a3-a30b-e2034e60b016 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:53.503850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 561/1110] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:38:53 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:38:55.568241 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ae816-5c88-44b1-aabe-67c6435f935d None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:55.569120 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ae816-5c88-44b1-aabe-67c6435f935d None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:55.569120 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ae816-5c88-44b1-aabe-67c6435f935d None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:55.570497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 550/1111] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:55.577604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cd06a-79ff-4cdb-91bd-6c3a9c843a57 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:55.578115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cd06a-79ff-4cdb-91bd-6c3a9c843a57 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:55.578214 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cd06a-79ff-4cdb-91bd-6c3a9c843a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:55.595883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-009cd06a-79ff-4cdb-91bd-6c3a9c843a57 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:55.597116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-009cd06a-79ff-4cdb-91bd-6c3a9c843a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:55.654497 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:55.663891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 562/1112] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:38:55.687385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ae816-5c88-44b1-aabe-67c6435f935d None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:55.788213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:55.792377 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ae816-5c88-44b1-aabe-67c6435f935d None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:55.825656 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97863d97-31da-4c54-948d-57ec30c1f343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:55.826230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97863d97-31da-4c54-948d-57ec30c1f343 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:55.826514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97863d97-31da-4c54-948d-57ec30c1f343 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:55.826792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97863d97-31da-4c54-948d-57ec30c1f343 None admin] PATH_INFO: `/v3/roles/548e622a9ae745d99d4cd34e68c1652c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:55.828220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97863d97-31da-4c54-948d-57ec30c1f343 None admin] RBAC: Authorizing `identity:get_role(role_id=548e622a9ae745d99d4cd34e68c1652c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:55.829619 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97863d97-31da-4c54-948d-57ec30c1f343 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:55.830969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 551/1113] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:38:55 2022] GET /identity/v3/roles/548e622a9ae745d99d4cd34e68c1652c => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:55.836976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cd06a-79ff-4cdb-91bd-6c3a9c843a57 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:55.935385 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cd06a-79ff-4cdb-91bd-6c3a9c843a57 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:55.968435 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56181f8a-86fb-49f1-b9fc-fe73bbfe7778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:55.969069 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56181f8a-86fb-49f1-b9fc-fe73bbfe7778 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:55.969347 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56181f8a-86fb-49f1-b9fc-fe73bbfe7778 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:55.969619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56181f8a-86fb-49f1-b9fc-fe73bbfe7778 None admin] PATH_INFO: `/v3/users/daa9b1eb5aea4d159c86c6eee0c67a49` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:55.983918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56181f8a-86fb-49f1-b9fc-fe73bbfe7778 None admin] RBAC: Authorizing `identity:get_user(user_id=daa9b1eb5aea4d159c86c6eee0c67a49)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:55.985387 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56181f8a-86fb-49f1-b9fc-fe73bbfe7778 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:55.987048 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 563/1114] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:38:55 2022] GET /identity/v3/users/daa9b1eb5aea4d159c86c6eee0c67a49 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:55.993669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97863d97-31da-4c54-948d-57ec30c1f343 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:56.090212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97863d97-31da-4c54-948d-57ec30c1f343 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:56.124021 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d62ea-d983-4605-9b4c-c79aa334caba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:56.124667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d62ea-d983-4605-9b4c-c79aa334caba None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:56.124952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d62ea-d983-4605-9b4c-c79aa334caba None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:56.125226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d62ea-d983-4605-9b4c-c79aa334caba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:56.159249 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:56.194646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d62ea-d983-4605-9b4c-c79aa334caba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:56.196084 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d62ea-d983-4605-9b4c-c79aa334caba None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:56.292696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 552/1115] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:38:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:38:56.298722 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56181f8a-86fb-49f1-b9fc-fe73bbfe7778 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:56.350602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:56.397072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56181f8a-86fb-49f1-b9fc-fe73bbfe7778 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:56.431631 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060884e3-4709-4c58-abd3-da3b1bf6ac77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:56.432265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060884e3-4709-4c58-abd3-da3b1bf6ac77 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:56.432572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060884e3-4709-4c58-abd3-da3b1bf6ac77 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:56.432847 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060884e3-4709-4c58-abd3-da3b1bf6ac77 None admin] PATH_INFO: `/v3/projects/ab79d54bb93d433e90bd44b897dd4c57` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:56.433998 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060884e3-4709-4c58-abd3-da3b1bf6ac77 None admin] RBAC: Authorizing `identity:get_project(project_id=ab79d54bb93d433e90bd44b897dd4c57)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:56.435441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060884e3-4709-4c58-abd3-da3b1bf6ac77 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:56.437126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 564/1116] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:38:56 2022] GET /identity/v3/projects/ab79d54bb93d433e90bd44b897dd4c57 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:56.443754 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d62ea-d983-4605-9b4c-c79aa334caba None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:56.512528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:56.546035 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d62ea-d983-4605-9b4c-c79aa334caba None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:56.580145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8472248-eb86-46f8-82af-c76f69d0ffef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:56.580815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8472248-eb86-46f8-82af-c76f69d0ffef None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:56.581049 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8472248-eb86-46f8-82af-c76f69d0ffef None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:56.581318 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8472248-eb86-46f8-82af-c76f69d0ffef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:56.582102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8472248-eb86-46f8-82af-c76f69d0ffef None admin] RBAC: Adding query filter params (role.id=548e622a9ae745d99d4cd34e68c1652c, scope.project.id=ab79d54bb93d433e90bd44b897dd4c57, user.id=daa9b1eb5aea4d159c86c6eee0c67a49) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:38:56.582408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8472248-eb86-46f8-82af-c76f69d0ffef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:56.584969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8472248-eb86-46f8-82af-c76f69d0ffef None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:56.590224 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 553/1117] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:38:56 2022] GET /identity/v3/role_assignments?user.id=daa9b1eb5aea4d159c86c6eee0c67a49&scope.project.id=ab79d54bb93d433e90bd44b897dd4c57&role.id=548e622a9ae745d99d4cd34e68c1652c => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:58.609004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b6615-9e8c-4f9b-b948-f5934bd81b79 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:58.609560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b6615-9e8c-4f9b-b948-f5934bd81b79 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:58.609560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b6615-9e8c-4f9b-b948-f5934bd81b79 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:58.610901 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 565/1118] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:38:58.616423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaf24c-3c68-4462-8f1c-3d440893e833 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:58.616858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaf24c-3c68-4462-8f1c-3d440893e833 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:58.616961 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaf24c-3c68-4462-8f1c-3d440893e833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:58.634843 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-85aaf24c-3c68-4462-8f1c-3d440893e833 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:38:58.636090 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-85aaf24c-3c68-4462-8f1c-3d440893e833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:38:58.700657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 554/1119] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:38:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:38:58.722631 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b6615-9e8c-4f9b-b948-f5934bd81b79 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:58.824352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b6615-9e8c-4f9b-b948-f5934bd81b79 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:58.844738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:58.858932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444d2fb-6ce1-4cf5-975c-5c42c25821fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:58.859615 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444d2fb-6ce1-4cf5-975c-5c42c25821fd None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:58.859850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444d2fb-6ce1-4cf5-975c-5c42c25821fd None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:58.860127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444d2fb-6ce1-4cf5-975c-5c42c25821fd None admin] PATH_INFO: `/v3/roles/548e622a9ae745d99d4cd34e68c1652c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:58.861603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444d2fb-6ce1-4cf5-975c-5c42c25821fd None admin] RBAC: Authorizing `identity:get_role(role_id=548e622a9ae745d99d4cd34e68c1652c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:58.862969 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444d2fb-6ce1-4cf5-975c-5c42c25821fd None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:58.865127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 566/1120] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:38:58 2022] GET /identity/v3/roles/548e622a9ae745d99d4cd34e68c1652c => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:58.871037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aaf24c-3c68-4462-8f1c-3d440893e833 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:58.931354 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:58.970443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aaf24c-3c68-4462-8f1c-3d440893e833 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:59.003086 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5930e-ed8b-4892-adac-62ebfb71c178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:59.003726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5930e-ed8b-4892-adac-62ebfb71c178 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:59.004011 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5930e-ed8b-4892-adac-62ebfb71c178 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:59.004302 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5930e-ed8b-4892-adac-62ebfb71c178 None admin] PATH_INFO: `/v3/users/daa9b1eb5aea4d159c86c6eee0c67a49` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:59.006111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5930e-ed8b-4892-adac-62ebfb71c178 None admin] RBAC: Authorizing `identity:get_user(user_id=daa9b1eb5aea4d159c86c6eee0c67a49)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:59.007534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5930e-ed8b-4892-adac-62ebfb71c178 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:59.009175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 555/1121] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:38:58 2022] GET /identity/v3/users/daa9b1eb5aea4d159c86c6eee0c67a49 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:59.014989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444d2fb-6ce1-4cf5-975c-5c42c25821fd None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:59.114158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444d2fb-6ce1-4cf5-975c-5c42c25821fd None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:59.147640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fc800-83ae-4cce-89a4-b7cef191a36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:59.148273 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fc800-83ae-4cce-89a4-b7cef191a36e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:59.148583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fc800-83ae-4cce-89a4-b7cef191a36e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:59.148858 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fc800-83ae-4cce-89a4-b7cef191a36e None admin] PATH_INFO: `/v3/projects/ab79d54bb93d433e90bd44b897dd4c57` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:59.150001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fc800-83ae-4cce-89a4-b7cef191a36e None admin] RBAC: Authorizing `identity:get_project(project_id=ab79d54bb93d433e90bd44b897dd4c57)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:59.151423 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fc800-83ae-4cce-89a4-b7cef191a36e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:59.153179 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 567/1122] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:38:59 2022] GET /identity/v3/projects/ab79d54bb93d433e90bd44b897dd4c57 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:38:59.159381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5930e-ed8b-4892-adac-62ebfb71c178 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:38:59.260504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5930e-ed8b-4892-adac-62ebfb71c178 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:38:59.288102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:38:59.294335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101d90a-32fe-40c0-816a-357db1cd0590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:38:59.295007 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101d90a-32fe-40c0-816a-357db1cd0590 None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:38:59.295292 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101d90a-32fe-40c0-816a-357db1cd0590 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:38:59.295571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101d90a-32fe-40c0-816a-357db1cd0590 None admin] PATH_INFO: `/v3/projects/ab79d54bb93d433e90bd44b897dd4c57/users/daa9b1eb5aea4d159c86c6eee0c67a49/roles/548e622a9ae745d99d4cd34e68c1652c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:38:59.297779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101d90a-32fe-40c0-816a-357db1cd0590 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab79d54bb93d433e90bd44b897dd4c57, user_id=daa9b1eb5aea4d159c86c6eee0c67a49, role_id=548e622a9ae745d99d4cd34e68c1652c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:38:59.299204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101d90a-32fe-40c0-816a-357db1cd0590 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:38:59.312438 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 556/1123] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:38:59 2022] PUT /identity/v3/projects/ab79d54bb93d433e90bd44b897dd4c57/users/daa9b1eb5aea4d159c86c6eee0c67a49/roles/548e622a9ae745d99d4cd34e68c1652c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:39:01.339405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e16e7-4364-4d77-8c9b-602f75c2342a None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:01.339964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e16e7-4364-4d77-8c9b-602f75c2342a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:01.339964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e16e7-4364-4d77-8c9b-602f75c2342a None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:01.341352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 568/1124] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:39:01.346664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d74b79-28af-44c2-9027-e97dee135f81 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:01.347091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d74b79-28af-44c2-9027-e97dee135f81 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:01.347219 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d74b79-28af-44c2-9027-e97dee135f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:01.365152 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-53d74b79-28af-44c2-9027-e97dee135f81 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:39:01.366365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-53d74b79-28af-44c2-9027-e97dee135f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:39:01.434877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 557/1125] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:39:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:39:01.458361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e16e7-4364-4d77-8c9b-602f75c2342a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:01.516296 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:01.561670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e16e7-4364-4d77-8c9b-602f75c2342a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:01.595271 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114ccf7-37f3-4abb-9515-47b0cb25f6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:01.595844 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114ccf7-37f3-4abb-9515-47b0cb25f6ee None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:01.596127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114ccf7-37f3-4abb-9515-47b0cb25f6ee None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:01.596434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114ccf7-37f3-4abb-9515-47b0cb25f6ee None admin] PATH_INFO: `/v3/roles/548e622a9ae745d99d4cd34e68c1652c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:01.597871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114ccf7-37f3-4abb-9515-47b0cb25f6ee None admin] RBAC: Authorizing `identity:get_role(role_id=548e622a9ae745d99d4cd34e68c1652c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:01.599280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114ccf7-37f3-4abb-9515-47b0cb25f6ee None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:01.600887 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 569/1126] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:39:01 2022] GET /identity/v3/roles/548e622a9ae745d99d4cd34e68c1652c => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:01.606752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d74b79-28af-44c2-9027-e97dee135f81 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:01.704419 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d74b79-28af-44c2-9027-e97dee135f81 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:01.724164 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:01.738602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b667f79-5acd-4626-a83e-f5dcae76ba36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:01.739930 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b667f79-5acd-4626-a83e-f5dcae76ba36 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:01.740269 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b667f79-5acd-4626-a83e-f5dcae76ba36 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:01.740582 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b667f79-5acd-4626-a83e-f5dcae76ba36 None admin] PATH_INFO: `/v3/users/daa9b1eb5aea4d159c86c6eee0c67a49` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:01.741752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b667f79-5acd-4626-a83e-f5dcae76ba36 None admin] RBAC: Authorizing `identity:get_user(user_id=daa9b1eb5aea4d159c86c6eee0c67a49)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:01.743172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b667f79-5acd-4626-a83e-f5dcae76ba36 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:01.744822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 558/1127] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:39:01 2022] GET /identity/v3/users/daa9b1eb5aea4d159c86c6eee0c67a49 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:01.750641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114ccf7-37f3-4abb-9515-47b0cb25f6ee None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:01.847659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114ccf7-37f3-4abb-9515-47b0cb25f6ee None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:01.870552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:01.882127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a70f6-c2f9-4f9d-9676-b9a617cafae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:01.882737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a70f6-c2f9-4f9d-9676-b9a617cafae8 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:01.883014 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a70f6-c2f9-4f9d-9676-b9a617cafae8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:01.883285 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a70f6-c2f9-4f9d-9676-b9a617cafae8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:01.952595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a70f6-c2f9-4f9d-9676-b9a617cafae8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:01.954061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a70f6-c2f9-4f9d-9676-b9a617cafae8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:02.053179 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 570/1128] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:39:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 18:39:02.059086 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b667f79-5acd-4626-a83e-f5dcae76ba36 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:02.158716 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b667f79-5acd-4626-a83e-f5dcae76ba36 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:02.190978 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc573c28-7fca-4f67-815c-2858d5e66887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:02.191611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc573c28-7fca-4f67-815c-2858d5e66887 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:02.191892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc573c28-7fca-4f67-815c-2858d5e66887 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:02.193085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc573c28-7fca-4f67-815c-2858d5e66887 None admin] PATH_INFO: `/v3/projects/ab79d54bb93d433e90bd44b897dd4c57` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:02.194209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc573c28-7fca-4f67-815c-2858d5e66887 None admin] RBAC: Authorizing `identity:get_project(project_id=ab79d54bb93d433e90bd44b897dd4c57)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:02.195643 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc573c28-7fca-4f67-815c-2858d5e66887 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:02.197332 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 559/1129] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:39:02 2022] GET /identity/v3/projects/ab79d54bb93d433e90bd44b897dd4c57 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:02.203484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a70f6-c2f9-4f9d-9676-b9a617cafae8 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:02.228771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:02.306975 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a70f6-c2f9-4f9d-9676-b9a617cafae8 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:02.343409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf7fb5-44e5-4f26-a244-69aafd623f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:02.344137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf7fb5-44e5-4f26-a244-69aafd623f86 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:02.344342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf7fb5-44e5-4f26-a244-69aafd623f86 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:02.344621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf7fb5-44e5-4f26-a244-69aafd623f86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:02.345423 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf7fb5-44e5-4f26-a244-69aafd623f86 None admin] RBAC: Adding query filter params (role.id=548e622a9ae745d99d4cd34e68c1652c, scope.project.id=ab79d54bb93d433e90bd44b897dd4c57, user.id=daa9b1eb5aea4d159c86c6eee0c67a49) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:39:02.345737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf7fb5-44e5-4f26-a244-69aafd623f86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:02.347184 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf7fb5-44e5-4f26-a244-69aafd623f86 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:02.353986 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 571/1130] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:39:02 2022] GET /identity/v3/role_assignments?user.id=daa9b1eb5aea4d159c86c6eee0c67a49&scope.project.id=ab79d54bb93d433e90bd44b897dd4c57&role.id=548e622a9ae745d99d4cd34e68c1652c => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:04.386854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4d49d-fe0a-4b44-821e-818477984035 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:04.387422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4d49d-fe0a-4b44-821e-818477984035 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:04.387422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4d49d-fe0a-4b44-821e-818477984035 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:04.388796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 560/1131] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:39:04.394001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abf3ed-5d61-4196-8426-ab97f430d619 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:04.394411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abf3ed-5d61-4196-8426-ab97f430d619 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:04.394563 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abf3ed-5d61-4196-8426-ab97f430d619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:04.413112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-10abf3ed-5d61-4196-8426-ab97f430d619 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:39:04.414319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-10abf3ed-5d61-4196-8426-ab97f430d619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:39:04.484711 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 572/1132] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:39:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:39:04.511857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4d49d-fe0a-4b44-821e-818477984035 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:04.568130 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:04.612107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4d49d-fe0a-4b44-821e-818477984035 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:04.646033 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbca14a-87a5-40a5-b8d5-070a6aebd8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:04.646592 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbca14a-87a5-40a5-b8d5-070a6aebd8f6 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:04.646886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbca14a-87a5-40a5-b8d5-070a6aebd8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:04.647141 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbca14a-87a5-40a5-b8d5-070a6aebd8f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:04.648275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbca14a-87a5-40a5-b8d5-070a6aebd8f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:04.649716 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbca14a-87a5-40a5-b8d5-070a6aebd8f6 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:04.651280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 561/1133] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:39:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:04.657159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abf3ed-5d61-4196-8426-ab97f430d619 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:04.675113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:04.758954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abf3ed-5d61-4196-8426-ab97f430d619 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:04.792958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd8b40-7f1a-421f-b748-b3cf9454d54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:04.793593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd8b40-7f1a-421f-b748-b3cf9454d54d None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:04.793878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd8b40-7f1a-421f-b748-b3cf9454d54d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:04.794152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd8b40-7f1a-421f-b748-b3cf9454d54d None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:04.794857 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd8b40-7f1a-421f-b748-b3cf9454d54d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:04.796332 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd8b40-7f1a-421f-b748-b3cf9454d54d None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:04.826600 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 573/1134] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Aug 3 18:39:04 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:39:06.877743 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f65a0a-63db-4b0a-95c9-5e1dda465638 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:06.878280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f65a0a-63db-4b0a-95c9-5e1dda465638 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:06.878280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f65a0a-63db-4b0a-95c9-5e1dda465638 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:06.880272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 562/1135] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:39:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:39:06.885625 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902bf7f-97b9-4faf-92d4-afe98a79d37f None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:06.886052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902bf7f-97b9-4faf-92d4-afe98a79d37f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:06.886186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902bf7f-97b9-4faf-92d4-afe98a79d37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:06.903928 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-f902bf7f-97b9-4faf-92d4-afe98a79d37f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:39:06.905173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f902bf7f-97b9-4faf-92d4-afe98a79d37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:39:06.966216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:06.971409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 574/1136] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:39:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:39:06.995852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f65a0a-63db-4b0a-95c9-5e1dda465638 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:07.096406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f65a0a-63db-4b0a-95c9-5e1dda465638 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:07.120196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:07.132024 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081ec10-4305-4d53-bd99-1503822d2ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:07.132631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081ec10-4305-4d53-bd99-1503822d2ad9 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:07.132916 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081ec10-4305-4d53-bd99-1503822d2ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:07.133189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081ec10-4305-4d53-bd99-1503822d2ad9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:07.134333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081ec10-4305-4d53-bd99-1503822d2ad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:07.135755 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081ec10-4305-4d53-bd99-1503822d2ad9 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:07.137379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 563/1137] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:39:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:07.143088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902bf7f-97b9-4faf-92d4-afe98a79d37f None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:07.240187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902bf7f-97b9-4faf-92d4-afe98a79d37f None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:07.273460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6bcb1-f976-410a-b172-bc198441d589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:07.274073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6bcb1-f976-410a-b172-bc198441d589 None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:07.274356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6bcb1-f976-410a-b172-bc198441d589 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:07.274632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6bcb1-f976-410a-b172-bc198441d589 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:07.275321 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6bcb1-f976-410a-b172-bc198441d589 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:07.276789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6bcb1-f976-410a-b172-bc198441d589 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:07.297535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 575/1138] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:39:07 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:39:09.382972 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee46254-3b2d-4c10-85c6-e17ca4f6d641 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:09.383510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee46254-3b2d-4c10-85c6-e17ca4f6d641 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:09.383510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee46254-3b2d-4c10-85c6-e17ca4f6d641 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:09.384920 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 564/1139] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:39:09.390375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150b82b-efd6-4712-ba53-4971ec9b7d44 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:09.390794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150b82b-efd6-4712-ba53-4971ec9b7d44 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:09.390937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150b82b-efd6-4712-ba53-4971ec9b7d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:09.409440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-c150b82b-efd6-4712-ba53-4971ec9b7d44 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:39:09.410626 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c150b82b-efd6-4712-ba53-4971ec9b7d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:39:09.462073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:09.479359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 576/1140] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:39:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:39:09.503471 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee46254-3b2d-4c10-85c6-e17ca4f6d641 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:09.621747 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee46254-3b2d-4c10-85c6-e17ca4f6d641 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:09.657163 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084f320-2523-4c21-9f04-4cd25a66a74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:09.657713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084f320-2523-4c21-9f04-4cd25a66a74b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:09.657996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084f320-2523-4c21-9f04-4cd25a66a74b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:09.658269 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084f320-2523-4c21-9f04-4cd25a66a74b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:09.669119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084f320-2523-4c21-9f04-4cd25a66a74b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-c084f320-2523-4c21-9f04-4cd25a66a74b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:39:09.673955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:39:09.675146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:39:09.676260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:39:09.676260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:39:09.676260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:39:09.676260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:39:09.676260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:39:09.676260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:39:09.676260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:39:09.676260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:39:09.676260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:39:09.676260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:39:09.676260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:39:09.676260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:39:09.676260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 565/1141] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:39:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:39:09.682045 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150b82b-efd6-4712-ba53-4971ec9b7d44 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:09.794978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150b82b-efd6-4712-ba53-4971ec9b7d44 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:09.834158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c6dc8-dcf6-489a-91ad-4a5ef4f49527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:09.834798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c6dc8-dcf6-489a-91ad-4a5ef4f49527 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:09.835081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c6dc8-dcf6-489a-91ad-4a5ef4f49527 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:09.835356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c6dc8-dcf6-489a-91ad-4a5ef4f49527 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:09.836083 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c6dc8-dcf6-489a-91ad-4a5ef4f49527 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:39:09.836419 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c6dc8-dcf6-489a-91ad-4a5ef4f49527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:09.837808 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c6dc8-dcf6-489a-91ad-4a5ef4f49527 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:09.846671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 577/1142] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:39:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:09.853123 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084f320-2523-4c21-9f04-4cd25a66a74b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:09.908766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:09.964149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084f320-2523-4c21-9f04-4cd25a66a74b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:10.000921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2782495-c66d-4bf7-83f0-ca20ec28ae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:10.001577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2782495-c66d-4bf7-83f0-ca20ec28ae67 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:10.002115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2782495-c66d-4bf7-83f0-ca20ec28ae67 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:10.002389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2782495-c66d-4bf7-83f0-ca20ec28ae67 None admin] PATH_INFO: `/v3/users/f2353e74260c489a9cc008e5f3384985` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:10.017217 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2782495-c66d-4bf7-83f0-ca20ec28ae67 None admin] RBAC: Authorizing `identity:get_user(user_id=f2353e74260c489a9cc008e5f3384985)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:10.018946 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2782495-c66d-4bf7-83f0-ca20ec28ae67 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:10.020885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 566/1143] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:39:09 2022] GET /identity/v3/users/f2353e74260c489a9cc008e5f3384985 => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:10.030473 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c6dc8-dcf6-489a-91ad-4a5ef4f49527 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:10.086688 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:10.142594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c6dc8-dcf6-489a-91ad-4a5ef4f49527 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:10.179878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da8053-f06b-4dfa-91fc-82ac631b8b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:10.180440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da8053-f06b-4dfa-91fc-82ac631b8b57 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:10.180724 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da8053-f06b-4dfa-91fc-82ac631b8b57 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:10.180995 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da8053-f06b-4dfa-91fc-82ac631b8b57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:10.250872 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da8053-f06b-4dfa-91fc-82ac631b8b57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:10.252605 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da8053-f06b-4dfa-91fc-82ac631b8b57 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:10.298802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:10.349785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 578/1144] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:39:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:39:10.357381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2782495-c66d-4bf7-83f0-ca20ec28ae67 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:10.455675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2782495-c66d-4bf7-83f0-ca20ec28ae67 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:10.469312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:10.493840 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020628a2-3ec9-4a7a-be3f-5e9ee1437513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:10.494482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020628a2-3ec9-4a7a-be3f-5e9ee1437513 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:10.494760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020628a2-3ec9-4a7a-be3f-5e9ee1437513 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:10.495031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020628a2-3ec9-4a7a-be3f-5e9ee1437513 None admin] PATH_INFO: `/v3/projects/bfde3d2eab1d4fce8b5e349c50c01572` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:10.496174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020628a2-3ec9-4a7a-be3f-5e9ee1437513 None admin] RBAC: Authorizing `identity:get_project(project_id=bfde3d2eab1d4fce8b5e349c50c01572)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:10.497634 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020628a2-3ec9-4a7a-be3f-5e9ee1437513 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:10.499295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 567/1145] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:39:10 2022] GET /identity/v3/projects/bfde3d2eab1d4fce8b5e349c50c01572 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:10.509352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da8053-f06b-4dfa-91fc-82ac631b8b57 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:10.603659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da8053-f06b-4dfa-91fc-82ac631b8b57 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:10.639551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d16cf6-5635-41cd-9f0a-d0c82bdf4454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:10.640164 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d16cf6-5635-41cd-9f0a-d0c82bdf4454 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:10.640472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d16cf6-5635-41cd-9f0a-d0c82bdf4454 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:10.640742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d16cf6-5635-41cd-9f0a-d0c82bdf4454 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:10.641532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d16cf6-5635-41cd-9f0a-d0c82bdf4454 None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=bfde3d2eab1d4fce8b5e349c50c01572, user.id=f2353e74260c489a9cc008e5f3384985) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:39:10.641835 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d16cf6-5635-41cd-9f0a-d0c82bdf4454 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:10.643258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d16cf6-5635-41cd-9f0a-d0c82bdf4454 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:10.649496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 579/1146] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:39:10 2022] GET /identity/v3/role_assignments?user.id=f2353e74260c489a9cc008e5f3384985&scope.project.id=bfde3d2eab1d4fce8b5e349c50c01572&role.id=19557e941d6f475ba92cf862846f9872 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:39:12.623505 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edf0b6-4d03-4f41-bc12-df6bc23993aa None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:12.624066 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edf0b6-4d03-4f41-bc12-df6bc23993aa None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:12.624066 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edf0b6-4d03-4f41-bc12-df6bc23993aa None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:12.625494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 568/1147] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:39:12.631797 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa87d9f-8f71-43d6-9281-cd42c7cf33e8 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:12.632217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa87d9f-8f71-43d6-9281-cd42c7cf33e8 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:12.632374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa87d9f-8f71-43d6-9281-cd42c7cf33e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:12.649817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-2aa87d9f-8f71-43d6-9281-cd42c7cf33e8 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:39:12.650986 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2aa87d9f-8f71-43d6-9281-cd42c7cf33e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:39:12.658292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:12.717524 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 580/1148] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:39:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:39:12.740642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edf0b6-4d03-4f41-bc12-df6bc23993aa None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:12.840461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edf0b6-4d03-4f41-bc12-df6bc23993aa None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:12.873369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c8ad0-9d8a-43b9-9688-2798f5de715a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:12.874043 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c8ad0-9d8a-43b9-9688-2798f5de715a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:12.874270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c8ad0-9d8a-43b9-9688-2798f5de715a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:12.874538 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c8ad0-9d8a-43b9-9688-2798f5de715a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:12.886919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c8ad0-9d8a-43b9-9688-2798f5de715a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-0e1c8ad0-9d8a-43b9-9688-2798f5de715a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:39:12.891757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:39:12.892914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:39:12.893881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:39:12.893881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:39:12.893881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:39:12.893881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:39:12.893881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:39:12.893881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:39:12.893881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:39:12.893881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:39:12.893881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:39:12.893881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:39:12.893881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:39:12.893881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:39:12.893881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 569/1149] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:39:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:39:12.899202 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa87d9f-8f71-43d6-9281-cd42c7cf33e8 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:12.997026 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa87d9f-8f71-43d6-9281-cd42c7cf33e8 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:13.030095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19f886-837d-4a9f-afca-be156a628941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:13.030707 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19f886-837d-4a9f-afca-be156a628941 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:13.030995 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19f886-837d-4a9f-afca-be156a628941 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:13.031267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19f886-837d-4a9f-afca-be156a628941 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:13.031970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19f886-837d-4a9f-afca-be156a628941 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:39:13.032272 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19f886-837d-4a9f-afca-be156a628941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:13.033657 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19f886-837d-4a9f-afca-be156a628941 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:13.041921 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 581/1150] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:39:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:13.047807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c8ad0-9d8a-43b9-9688-2798f5de715a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:13.087317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:13.146532 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c8ad0-9d8a-43b9-9688-2798f5de715a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:13.180604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1849d-a08a-441e-afc8-eea19e5750e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:13.181236 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1849d-a08a-441e-afc8-eea19e5750e8 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:13.181516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1849d-a08a-441e-afc8-eea19e5750e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:13.181787 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1849d-a08a-441e-afc8-eea19e5750e8 None admin] PATH_INFO: `/v3/users/f2353e74260c489a9cc008e5f3384985` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:13.182945 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1849d-a08a-441e-afc8-eea19e5750e8 None admin] RBAC: Authorizing `identity:get_user(user_id=f2353e74260c489a9cc008e5f3384985)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:13.184384 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1849d-a08a-441e-afc8-eea19e5750e8 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:13.185956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 570/1151] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:39:13 2022] GET /identity/v3/users/f2353e74260c489a9cc008e5f3384985 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:13.191816 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19f886-837d-4a9f-afca-be156a628941 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:13.195713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:13.292873 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19f886-837d-4a9f-afca-be156a628941 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:13.325949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9262441-08eb-4e59-b184-1257bab7a813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:13.326609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9262441-08eb-4e59-b184-1257bab7a813 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:13.326904 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9262441-08eb-4e59-b184-1257bab7a813 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:13.327156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9262441-08eb-4e59-b184-1257bab7a813 None admin] PATH_INFO: `/v3/projects/bfde3d2eab1d4fce8b5e349c50c01572` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:13.328361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9262441-08eb-4e59-b184-1257bab7a813 None admin] RBAC: Authorizing `identity:get_project(project_id=bfde3d2eab1d4fce8b5e349c50c01572)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:13.329787 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9262441-08eb-4e59-b184-1257bab7a813 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:13.331436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 582/1152] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:39:13 2022] GET /identity/v3/projects/bfde3d2eab1d4fce8b5e349c50c01572 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:13.337844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1849d-a08a-441e-afc8-eea19e5750e8 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:13.436762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1849d-a08a-441e-afc8-eea19e5750e8 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:13.443539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:13.470930 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697916d4-1bee-4ec7-86bd-da3e3e8a58db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:13.471598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697916d4-1bee-4ec7-86bd-da3e3e8a58db None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:13.471883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697916d4-1bee-4ec7-86bd-da3e3e8a58db None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:13.472160 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697916d4-1bee-4ec7-86bd-da3e3e8a58db None admin] PATH_INFO: `/v3/projects/bfde3d2eab1d4fce8b5e349c50c01572/users/f2353e74260c489a9cc008e5f3384985/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:13.474789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697916d4-1bee-4ec7-86bd-da3e3e8a58db None admin] RBAC: Authorizing `identity:create_grant(project_id=bfde3d2eab1d4fce8b5e349c50c01572, user_id=f2353e74260c489a9cc008e5f3384985, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:13.476206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697916d4-1bee-4ec7-86bd-da3e3e8a58db None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:13.489928 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 571/1153] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:39:13 2022] PUT /identity/v3/projects/bfde3d2eab1d4fce8b5e349c50c01572/users/f2353e74260c489a9cc008e5f3384985/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:39:15.473989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906de7e9-085f-4e03-bc88-e3a241f413c4 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:15.474567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906de7e9-085f-4e03-bc88-e3a241f413c4 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:15.474567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906de7e9-085f-4e03-bc88-e3a241f413c4 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:15.475889 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 583/1154] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:39:15.481163 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e18e19-9534-4dc8-ad52-ba35e1923ee6 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:15.481606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e18e19-9534-4dc8-ad52-ba35e1923ee6 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:15.481714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e18e19-9534-4dc8-ad52-ba35e1923ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:15.499890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-d3e18e19-9534-4dc8-ad52-ba35e1923ee6 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:39:15.501088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d3e18e19-9534-4dc8-ad52-ba35e1923ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:39:15.566540 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 572/1155] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:39:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:39:15.590470 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906de7e9-085f-4e03-bc88-e3a241f413c4 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:15.660530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:15.691014 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906de7e9-085f-4e03-bc88-e3a241f413c4 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:15.725239 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf42e1-02f9-4c32-89e8-49cca9e19ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:15.725805 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf42e1-02f9-4c32-89e8-49cca9e19ee1 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:15.726090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf42e1-02f9-4c32-89e8-49cca9e19ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:15.726364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf42e1-02f9-4c32-89e8-49cca9e19ee1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:15.736177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf42e1-02f9-4c32-89e8-49cca9e19ee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-14cf42e1-02f9-4c32-89e8-49cca9e19ee1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:39:15.741595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:39:15.742713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:39:15.743583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:39:15.743583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:39:15.743583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:39:15.743583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:39:15.743583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:39:15.743583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:39:15.743583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:39:15.743583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:39:15.743583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:39:15.743583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:39:15.743583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:39:15.743583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:39:15.743583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 584/1156] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:39:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:39:15.748705 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e18e19-9534-4dc8-ad52-ba35e1923ee6 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:15.840960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:15.847768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e18e19-9534-4dc8-ad52-ba35e1923ee6 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:15.882184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accda101-69cd-49e8-91fd-ea457fa3023a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:15.882811 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accda101-69cd-49e8-91fd-ea457fa3023a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:15.883088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accda101-69cd-49e8-91fd-ea457fa3023a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:15.883356 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accda101-69cd-49e8-91fd-ea457fa3023a None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:15.884058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accda101-69cd-49e8-91fd-ea457fa3023a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:39:15.884936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accda101-69cd-49e8-91fd-ea457fa3023a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:15.885887 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accda101-69cd-49e8-91fd-ea457fa3023a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:15.893846 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 573/1157] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:39:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:15.899804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf42e1-02f9-4c32-89e8-49cca9e19ee1 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:15.997177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf42e1-02f9-4c32-89e8-49cca9e19ee1 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:16.026479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:16.031694 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930e6c5-1bd2-49bb-b40c-7cccf82eb2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:16.032352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930e6c5-1bd2-49bb-b40c-7cccf82eb2ac None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:16.032634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930e6c5-1bd2-49bb-b40c-7cccf82eb2ac None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:16.032903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930e6c5-1bd2-49bb-b40c-7cccf82eb2ac None admin] PATH_INFO: `/v3/users/f2353e74260c489a9cc008e5f3384985` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:16.034073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930e6c5-1bd2-49bb-b40c-7cccf82eb2ac None admin] RBAC: Authorizing `identity:get_user(user_id=f2353e74260c489a9cc008e5f3384985)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:16.036002 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930e6c5-1bd2-49bb-b40c-7cccf82eb2ac None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:16.037704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 585/1158] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:39:15 2022] GET /identity/v3/users/f2353e74260c489a9cc008e5f3384985 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:16.043628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accda101-69cd-49e8-91fd-ea457fa3023a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:16.141883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accda101-69cd-49e8-91fd-ea457fa3023a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:16.176155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f929ef5-3c96-4d69-88f8-b9f0228abd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:16.177216 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f929ef5-3c96-4d69-88f8-b9f0228abd84 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:16.177492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f929ef5-3c96-4d69-88f8-b9f0228abd84 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:16.177757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f929ef5-3c96-4d69-88f8-b9f0228abd84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:16.204127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:16.249373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f929ef5-3c96-4d69-88f8-b9f0228abd84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:16.250725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f929ef5-3c96-4d69-88f8-b9f0228abd84 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:16.349243 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 574/1159] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:39:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:39:16.355394 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930e6c5-1bd2-49bb-b40c-7cccf82eb2ac None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:16.454911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930e6c5-1bd2-49bb-b40c-7cccf82eb2ac None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:16.488090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aac8c3-35db-4fde-80a7-0b7a017d42d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:16.488822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aac8c3-35db-4fde-80a7-0b7a017d42d0 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:16.489290 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aac8c3-35db-4fde-80a7-0b7a017d42d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:16.489581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aac8c3-35db-4fde-80a7-0b7a017d42d0 None admin] PATH_INFO: `/v3/projects/bfde3d2eab1d4fce8b5e349c50c01572` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:16.490960 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aac8c3-35db-4fde-80a7-0b7a017d42d0 None admin] RBAC: Authorizing `identity:get_project(project_id=bfde3d2eab1d4fce8b5e349c50c01572)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:16.492481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aac8c3-35db-4fde-80a7-0b7a017d42d0 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:16.494193 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 586/1160] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:39:16 2022] GET /identity/v3/projects/bfde3d2eab1d4fce8b5e349c50c01572 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:16.500482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f929ef5-3c96-4d69-88f8-b9f0228abd84 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:16.559734 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:16.603518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f929ef5-3c96-4d69-88f8-b9f0228abd84 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:16.636944 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa2ad7-5e9a-4ea6-8833-10391129bb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:16.637548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa2ad7-5e9a-4ea6-8833-10391129bb60 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:16.637822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa2ad7-5e9a-4ea6-8833-10391129bb60 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:16.638089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa2ad7-5e9a-4ea6-8833-10391129bb60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:16.638867 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa2ad7-5e9a-4ea6-8833-10391129bb60 None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=bfde3d2eab1d4fce8b5e349c50c01572, user.id=f2353e74260c489a9cc008e5f3384985) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:39:16.639170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa2ad7-5e9a-4ea6-8833-10391129bb60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:16.640601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa2ad7-5e9a-4ea6-8833-10391129bb60 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:16.647186 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 575/1161] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:39:16 2022] GET /identity/v3/role_assignments?user.id=f2353e74260c489a9cc008e5f3384985&scope.project.id=bfde3d2eab1d4fce8b5e349c50c01572&role.id=19557e941d6f475ba92cf862846f9872 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:18.689070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5b718-3558-4152-a801-e9c07fd1f948 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:18.689639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5b718-3558-4152-a801-e9c07fd1f948 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:18.689639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5b718-3558-4152-a801-e9c07fd1f948 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:18.690948 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 587/1162] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:39:18.697375 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03e45a-1725-4ace-85d0-1648fbadde16 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:18.697783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03e45a-1725-4ace-85d0-1648fbadde16 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:18.697934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03e45a-1725-4ace-85d0-1648fbadde16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:18.715531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-5d03e45a-1725-4ace-85d0-1648fbadde16 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:39:18.716758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-5d03e45a-1725-4ace-85d0-1648fbadde16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:39:18.782002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 576/1163] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:39:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:39:18.804506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5b718-3558-4152-a801-e9c07fd1f948 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:18.872298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:18.907772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5b718-3558-4152-a801-e9c07fd1f948 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:18.943929 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996e98d-52a9-43db-be00-3e1ccceaf89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:18.944513 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996e98d-52a9-43db-be00-3e1ccceaf89d None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:18.944793 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996e98d-52a9-43db-be00-3e1ccceaf89d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:18.945065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996e98d-52a9-43db-be00-3e1ccceaf89d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:19.015687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996e98d-52a9-43db-be00-3e1ccceaf89d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:19.017175 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996e98d-52a9-43db-be00-3e1ccceaf89d None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:19.080432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:19.114281 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 588/1164] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:39:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:39:19.120310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03e45a-1725-4ace-85d0-1648fbadde16 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:19.172382 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:19.218965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03e45a-1725-4ace-85d0-1648fbadde16 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:19.252850 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e425986-97a0-4390-894d-d6690c3dcebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:19.253481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e425986-97a0-4390-894d-d6690c3dcebf None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:19.253763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e425986-97a0-4390-894d-d6690c3dcebf None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:19.254038 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e425986-97a0-4390-894d-d6690c3dcebf None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:19.261475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e425986-97a0-4390-894d-d6690c3dcebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:19.262905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e425986-97a0-4390-894d-d6690c3dcebf None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-7e425986-97a0-4390-894d-d6690c3dcebf None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:39:19.275251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:39:19.276521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:39:19.277727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:39:19.277727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:39:19.277727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:39:19.277727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:39:19.277727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:39:19.277727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:39:19.277727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:39:19.277727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:39:19.277727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:39:19.277727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:39:19.277727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 03 18:39:19.277727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:39:19.277727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 03 18:39:19.277727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:39:19.277727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 577/1165] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Aug 3 18:39:19 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:39:19.282430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996e98d-52a9-43db-be00-3e1ccceaf89d None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:19.380584 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996e98d-52a9-43db-be00-3e1ccceaf89d None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:19.413970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14f22c-6e1a-4869-95c0-7f4fc4f457c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:19.414582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14f22c-6e1a-4869-95c0-7f4fc4f457c4 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:19.414883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14f22c-6e1a-4869-95c0-7f4fc4f457c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:19.415139 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14f22c-6e1a-4869-95c0-7f4fc4f457c4 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:19.415824 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14f22c-6e1a-4869-95c0-7f4fc4f457c4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:39:19.416126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14f22c-6e1a-4869-95c0-7f4fc4f457c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:19.417518 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14f22c-6e1a-4869-95c0-7f4fc4f457c4 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:19.424455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 589/1166] 10.222.0.69 () {56 vars in 1115 bytes} [Wed Aug 3 18:39:19 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:21.413708 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8be10-1022-41a7-aa12-0606187e9fa2 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:21.414274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8be10-1022-41a7-aa12-0606187e9fa2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:21.414274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8be10-1022-41a7-aa12-0606187e9fa2 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:21.415605 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 578/1167] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:39:21.420808 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ca7ec-8f3d-4f12-9643-d1770fa94f1b None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:21.421226 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ca7ec-8f3d-4f12-9643-d1770fa94f1b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:21.421363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ca7ec-8f3d-4f12-9643-d1770fa94f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:21.439629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-f08ca7ec-8f3d-4f12-9643-d1770fa94f1b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:39:21.440847 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f08ca7ec-8f3d-4f12-9643-d1770fa94f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:39:21.467244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:21.508886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 590/1168] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:39:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:39:21.531296 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8be10-1022-41a7-aa12-0606187e9fa2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:21.630353 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8be10-1022-41a7-aa12-0606187e9fa2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:21.642843 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:21.665952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f3389-de47-48a5-93c8-2d3092997e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:21.666784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f3389-de47-48a5-93c8-2d3092997e31 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:21.667208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f3389-de47-48a5-93c8-2d3092997e31 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:21.667618 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f3389-de47-48a5-93c8-2d3092997e31 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:21.668745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f3389-de47-48a5-93c8-2d3092997e31 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:21.670868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f3389-de47-48a5-93c8-2d3092997e31 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:21.689171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 579/1169] 10.222.0.69 () {60 vars in 1138 bytes} [Wed Aug 3 18:39:21 2022] POST /identity/v3/domains => generated 267 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:39:23.718617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bd534-0d10-4bb1-8bd9-2f5fb7db1491 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:23.719153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bd534-0d10-4bb1-8bd9-2f5fb7db1491 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:23.719153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bd534-0d10-4bb1-8bd9-2f5fb7db1491 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:23.720581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 591/1170] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:39:23.726431 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d3172-9475-4b20-bbc2-8316ada65a83 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:23.726872 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d3172-9475-4b20-bbc2-8316ada65a83 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:23.726998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d3172-9475-4b20-bbc2-8316ada65a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:23.744894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-8a3d3172-9475-4b20-bbc2-8316ada65a83 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:39:23.746076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8a3d3172-9475-4b20-bbc2-8316ada65a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:39:23.812085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 580/1171] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:39:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:39:23.836517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bd534-0d10-4bb1-8bd9-2f5fb7db1491 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:23.938572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bd534-0d10-4bb1-8bd9-2f5fb7db1491 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:23.972170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bdd7a3-5e0c-417e-9fb1-5138f2a89735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:23.972772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdd7a3-5e0c-417e-9fb1-5138f2a89735 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:23.973056 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdd7a3-5e0c-417e-9fb1-5138f2a89735 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:23.973331 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdd7a3-5e0c-417e-9fb1-5138f2a89735 None admin] PATH_INFO: `/v3/domains/a8374e057f1c4a459a3eef1db29eb17f` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:23.983633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bdd7a3-5e0c-417e-9fb1-5138f2a89735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a8374e057f1c4a459a3eef1db29eb17f)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:23.985135 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bdd7a3-5e0c-417e-9fb1-5138f2a89735 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:23.986743 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 592/1172] 10.222.0.69 () {56 vars in 1183 bytes} [Wed Aug 3 18:39:23 2022] GET /identity/v3/domains/a8374e057f1c4a459a3eef1db29eb17f => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:23.993678 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d3172-9475-4b20-bbc2-8316ada65a83 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:24.090977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d3172-9475-4b20-bbc2-8316ada65a83 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:24.092478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:24.126794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987993db-b53f-4d38-b0d6-efaac35d0de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:24.127401 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987993db-b53f-4d38-b0d6-efaac35d0de6 None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:24.127678 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987993db-b53f-4d38-b0d6-efaac35d0de6 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:24.127947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987993db-b53f-4d38-b0d6-efaac35d0de6 None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:24.128653 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987993db-b53f-4d38-b0d6-efaac35d0de6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:24.131212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987993db-b53f-4d38-b0d6-efaac35d0de6 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:24.156879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 581/1173] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Aug 3 18:39:23 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:39:26.194679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f94ac1-cc63-4bac-93c1-fe2f654dee6c None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:26.195187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f94ac1-cc63-4bac-93c1-fe2f654dee6c None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:26.195187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f94ac1-cc63-4bac-93c1-fe2f654dee6c None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:26.196498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 593/1174] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:39:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:39:26.201830 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431d8fb-c50c-415d-b9b7-53784b3e3f26 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:26.202260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431d8fb-c50c-415d-b9b7-53784b3e3f26 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:26.202393 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431d8fb-c50c-415d-b9b7-53784b3e3f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:26.220258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-e431d8fb-c50c-415d-b9b7-53784b3e3f26 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:39:26.221463 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e431d8fb-c50c-415d-b9b7-53784b3e3f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:39:26.288215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 582/1175] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:39:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:39:26.315539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f94ac1-cc63-4bac-93c1-fe2f654dee6c None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:26.342674 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:26.423396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f94ac1-cc63-4bac-93c1-fe2f654dee6c None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:26.457321 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7e773-7377-4962-a040-93abd36fc995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:26.458450 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7e773-7377-4962-a040-93abd36fc995 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:26.458733 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7e773-7377-4962-a040-93abd36fc995 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:26.459007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7e773-7377-4962-a040-93abd36fc995 None admin] PATH_INFO: `/v3/domains/a8374e057f1c4a459a3eef1db29eb17f` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:26.460152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7e773-7377-4962-a040-93abd36fc995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a8374e057f1c4a459a3eef1db29eb17f)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:26.461775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7e773-7377-4962-a040-93abd36fc995 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:26.463348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 594/1176] 10.222.0.69 () {56 vars in 1183 bytes} [Wed Aug 3 18:39:26 2022] GET /identity/v3/domains/a8374e057f1c4a459a3eef1db29eb17f => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:26.469269 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431d8fb-c50c-415d-b9b7-53784b3e3f26 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:26.554268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:26.570780 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431d8fb-c50c-415d-b9b7-53784b3e3f26 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:26.604236 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcc01c-56a6-4a36-af5a-efeacddc8cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:26.604877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcc01c-56a6-4a36-af5a-efeacddc8cde None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:26.605155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcc01c-56a6-4a36-af5a-efeacddc8cde None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:26.605421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcc01c-56a6-4a36-af5a-efeacddc8cde None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:26.606095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadcc01c-56a6-4a36-af5a-efeacddc8cde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:26.607498 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadcc01c-56a6-4a36-af5a-efeacddc8cde None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:26.629196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 583/1177] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:39:26 2022] POST /identity/v3/users => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:39:28.673086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc7770-82b6-457f-ab46-d17d85fc1561 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:28.673646 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc7770-82b6-457f-ab46-d17d85fc1561 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:28.673646 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc7770-82b6-457f-ab46-d17d85fc1561 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:28.674985 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 595/1178] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:39:28.680503 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744b1d1-55eb-4c39-a758-c557910a587f None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:28.680917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744b1d1-55eb-4c39-a758-c557910a587f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:28.681066 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744b1d1-55eb-4c39-a758-c557910a587f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:28.699528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-6744b1d1-55eb-4c39-a758-c557910a587f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:39:28.700740 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-6744b1d1-55eb-4c39-a758-c557910a587f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:39:28.766396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 584/1179] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:39:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:39:28.790418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc7770-82b6-457f-ab46-d17d85fc1561 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:28.881322 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:28.892113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc7770-82b6-457f-ab46-d17d85fc1561 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:28.926589 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd861a-5886-4e66-97a9-149c72302fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:28.927152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd861a-5886-4e66-97a9-149c72302fca None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:28.927434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd861a-5886-4e66-97a9-149c72302fca None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:28.927708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd861a-5886-4e66-97a9-149c72302fca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:28.937724 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd861a-5886-4e66-97a9-149c72302fca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-90dd861a-5886-4e66-97a9-149c72302fca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:39:28.943348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:39:28.944616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:39:28.945736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:39:28.945736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:39:28.945736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:39:28.945736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:39:28.945736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:39:28.945736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:39:28.945736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:39:28.945736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:39:28.945736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:39:28.945736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:39:28.945736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:39:28.945736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:39:28.945736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 596/1180] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:39:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:39:28.950667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744b1d1-55eb-4c39-a758-c557910a587f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:29.018909 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:29.050464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744b1d1-55eb-4c39-a758-c557910a587f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:29.085210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3be6e-f17c-4787-80e4-f97246939c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:29.085822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3be6e-f17c-4787-80e4-f97246939c3b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:29.086107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3be6e-f17c-4787-80e4-f97246939c3b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:29.086384 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3be6e-f17c-4787-80e4-f97246939c3b None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:29.087101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3be6e-f17c-4787-80e4-f97246939c3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:39:29.087409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3be6e-f17c-4787-80e4-f97246939c3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:29.088834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3be6e-f17c-4787-80e4-f97246939c3b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:29.096629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 585/1181] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:39:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:29.102851 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd861a-5886-4e66-97a9-149c72302fca None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:29.202049 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd861a-5886-4e66-97a9-149c72302fca None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:29.234815 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6142ac-1674-4972-a474-67b23a6265b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:29.235447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6142ac-1674-4972-a474-67b23a6265b8 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:29.235729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6142ac-1674-4972-a474-67b23a6265b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:29.236003 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6142ac-1674-4972-a474-67b23a6265b8 None admin] PATH_INFO: `/v3/users/4c7c58a2db2d4c72a7b39828c7171774` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:29.249405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6142ac-1674-4972-a474-67b23a6265b8 None admin] RBAC: Authorizing `identity:get_user(user_id=4c7c58a2db2d4c72a7b39828c7171774)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:29.250849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6142ac-1674-4972-a474-67b23a6265b8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:29.252744 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 597/1182] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:39:29 2022] GET /identity/v3/users/4c7c58a2db2d4c72a7b39828c7171774 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:29.259794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3be6e-f17c-4787-80e4-f97246939c3b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:29.359037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3be6e-f17c-4787-80e4-f97246939c3b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:29.391538 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:29.397699 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e47d40-21b1-47cf-9be2-c12ac5ceecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:29.398319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e47d40-21b1-47cf-9be2-c12ac5ceecd3 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:29.398608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e47d40-21b1-47cf-9be2-c12ac5ceecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:29.398873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e47d40-21b1-47cf-9be2-c12ac5ceecd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:29.466403 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e47d40-21b1-47cf-9be2-c12ac5ceecd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:29.467763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e47d40-21b1-47cf-9be2-c12ac5ceecd3 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:29.565304 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 586/1183] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:39:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:39:29.571895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6142ac-1674-4972-a474-67b23a6265b8 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:29.582436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:29.670766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6142ac-1674-4972-a474-67b23a6265b8 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:29.706132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292077b-6099-4fca-bba5-e02485c52772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:29.706920 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292077b-6099-4fca-bba5-e02485c52772 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:29.707028 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292077b-6099-4fca-bba5-e02485c52772 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:29.707304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292077b-6099-4fca-bba5-e02485c52772 None admin] PATH_INFO: `/v3/projects/1f0bc8792f574431a0dd1dc519ec89e2` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:29.709394 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292077b-6099-4fca-bba5-e02485c52772 None admin] RBAC: Authorizing `identity:get_project(project_id=1f0bc8792f574431a0dd1dc519ec89e2)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:29.710836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292077b-6099-4fca-bba5-e02485c52772 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:29.712541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 598/1184] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:39:29 2022] GET /identity/v3/projects/1f0bc8792f574431a0dd1dc519ec89e2 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:29.718751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e47d40-21b1-47cf-9be2-c12ac5ceecd3 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:29.748651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:29.817403 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e47d40-21b1-47cf-9be2-c12ac5ceecd3 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:29.851016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b561e6-2d96-4f19-a4a9-dc0ec139e8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:29.851614 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b561e6-2d96-4f19-a4a9-dc0ec139e8fe None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:29.851897 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b561e6-2d96-4f19-a4a9-dc0ec139e8fe None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:29.852173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b561e6-2d96-4f19-a4a9-dc0ec139e8fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:29.852998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b561e6-2d96-4f19-a4a9-dc0ec139e8fe None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=1f0bc8792f574431a0dd1dc519ec89e2, user.id=4c7c58a2db2d4c72a7b39828c7171774) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:39:29.853309 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b561e6-2d96-4f19-a4a9-dc0ec139e8fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:29.854728 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b561e6-2d96-4f19-a4a9-dc0ec139e8fe None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:29.860266 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 587/1185] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:39:29 2022] GET /identity/v3/role_assignments?user.id=4c7c58a2db2d4c72a7b39828c7171774&scope.project.id=1f0bc8792f574431a0dd1dc519ec89e2&role.id=19557e941d6f475ba92cf862846f9872 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:31.847235 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0162584-dc83-4da5-8342-b0fbac46300e None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:31.847789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0162584-dc83-4da5-8342-b0fbac46300e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:31.847789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0162584-dc83-4da5-8342-b0fbac46300e None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:31.849701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 599/1186] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:39:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:39:31.856892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5f06a-641f-4e5b-9606-16e59e9c75b7 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:31.857332 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5f06a-641f-4e5b-9606-16e59e9c75b7 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:31.857443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5f06a-641f-4e5b-9606-16e59e9c75b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:31.876073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-91d5f06a-641f-4e5b-9606-16e59e9c75b7 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:39:31.877561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-91d5f06a-641f-4e5b-9606-16e59e9c75b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:39:31.944480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 588/1187] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:39:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:39:31.967119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0162584-dc83-4da5-8342-b0fbac46300e None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:32.044448 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:32.069436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0162584-dc83-4da5-8342-b0fbac46300e None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:32.104397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2610d9d-c746-42a0-a63e-9f2e8580348e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:32.104992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2610d9d-c746-42a0-a63e-9f2e8580348e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:32.105278 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2610d9d-c746-42a0-a63e-9f2e8580348e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:32.105556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2610d9d-c746-42a0-a63e-9f2e8580348e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:32.117554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2610d9d-c746-42a0-a63e-9f2e8580348e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-f2610d9d-c746-42a0-a63e-9f2e8580348e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:39:32.122546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:39:32.123671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:39:32.124914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:39:32.124914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:39:32.124914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:39:32.124914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:39:32.124914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:39:32.124914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:39:32.124914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:39:32.124914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:39:32.124914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:39:32.124914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:39:32.124914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:39:32.124914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:39:32.124914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 600/1188] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:39:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:39:32.130491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5f06a-641f-4e5b-9606-16e59e9c75b7 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:32.159174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:32.232081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5f06a-641f-4e5b-9606-16e59e9c75b7 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:32.265074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00a229-a036-468f-a7db-cc3f91744c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:32.265686 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00a229-a036-468f-a7db-cc3f91744c1c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:32.265971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00a229-a036-468f-a7db-cc3f91744c1c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:32.266245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00a229-a036-468f-a7db-cc3f91744c1c None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:32.266955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00a229-a036-468f-a7db-cc3f91744c1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:39:32.267260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00a229-a036-468f-a7db-cc3f91744c1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:32.268648 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00a229-a036-468f-a7db-cc3f91744c1c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:32.277283 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 589/1189] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:39:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:39:32.283122 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2610d9d-c746-42a0-a63e-9f2e8580348e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:32.381790 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2610d9d-c746-42a0-a63e-9f2e8580348e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:32.416636 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04795454-3426-4a0c-94c2-f97a6576c90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:32.417261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04795454-3426-4a0c-94c2-f97a6576c90a None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:32.417541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04795454-3426-4a0c-94c2-f97a6576c90a None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:32.417809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04795454-3426-4a0c-94c2-f97a6576c90a None admin] PATH_INFO: `/v3/users/4c7c58a2db2d4c72a7b39828c7171774` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:32.418948 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04795454-3426-4a0c-94c2-f97a6576c90a None admin] RBAC: Authorizing `identity:get_user(user_id=4c7c58a2db2d4c72a7b39828c7171774)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:32.420414 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04795454-3426-4a0c-94c2-f97a6576c90a None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:32.421983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 601/1190] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:39:32 2022] GET /identity/v3/users/4c7c58a2db2d4c72a7b39828c7171774 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:32.427925 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00a229-a036-468f-a7db-cc3f91744c1c None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:32.523357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:32.535492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00a229-a036-468f-a7db-cc3f91744c1c None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:32.573369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5fdf1-2ec8-42c2-9d57-2b512f90552f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:32.574006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5fdf1-2ec8-42c2-9d57-2b512f90552f None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:32.574287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5fdf1-2ec8-42c2-9d57-2b512f90552f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:32.574559 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5fdf1-2ec8-42c2-9d57-2b512f90552f None admin] PATH_INFO: `/v3/projects/1f0bc8792f574431a0dd1dc519ec89e2` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:32.575700 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5fdf1-2ec8-42c2-9d57-2b512f90552f None admin] RBAC: Authorizing `identity:get_project(project_id=1f0bc8792f574431a0dd1dc519ec89e2)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:32.577149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5fdf1-2ec8-42c2-9d57-2b512f90552f None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:32.578816 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 590/1191] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:39:32 2022] GET /identity/v3/projects/1f0bc8792f574431a0dd1dc519ec89e2 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:32.585674 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04795454-3426-4a0c-94c2-f97a6576c90a None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:32.589535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:32.682979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04795454-3426-4a0c-94c2-f97a6576c90a None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:32.716152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2da47-73e6-42bb-b65c-4b246b000207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:32.716843 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2da47-73e6-42bb-b65c-4b246b000207 None admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:32.717129 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2da47-73e6-42bb-b65c-4b246b000207 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:32.717408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2da47-73e6-42bb-b65c-4b246b000207 None admin] PATH_INFO: `/v3/projects/1f0bc8792f574431a0dd1dc519ec89e2/users/4c7c58a2db2d4c72a7b39828c7171774/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:32.719315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2da47-73e6-42bb-b65c-4b246b000207 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f0bc8792f574431a0dd1dc519ec89e2, user_id=4c7c58a2db2d4c72a7b39828c7171774, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:32.720808 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2da47-73e6-42bb-b65c-4b246b000207 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:32.733539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 602/1192] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:39:32 2022] PUT /identity/v3/projects/1f0bc8792f574431a0dd1dc519ec89e2/users/4c7c58a2db2d4c72a7b39828c7171774/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:39:34.706802 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1885578-98e0-4da3-a1a8-b1b0ad58d2ea None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:34.707890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1885578-98e0-4da3-a1a8-b1b0ad58d2ea None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:34.708091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1885578-98e0-4da3-a1a8-b1b0ad58d2ea None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:34.709309 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 591/1193] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:39:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:39:34.714553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a98ff4-5d53-4731-936e-847b4ec93a2b None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:34.715002 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a98ff4-5d53-4731-936e-847b4ec93a2b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:34.715096 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a98ff4-5d53-4731-936e-847b4ec93a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:34.733836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-19a98ff4-5d53-4731-936e-847b4ec93a2b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:39:34.734994 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-19a98ff4-5d53-4731-936e-847b4ec93a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:39:34.794563 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:34.804170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 603/1194] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:39:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:39:34.827978 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1885578-98e0-4da3-a1a8-b1b0ad58d2ea None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:34.928020 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1885578-98e0-4da3-a1a8-b1b0ad58d2ea None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:34.962418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c406b8-bdc5-4278-a1de-d13aba2cff22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:34.962986 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c406b8-bdc5-4278-a1de-d13aba2cff22 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:34.963270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c406b8-bdc5-4278-a1de-d13aba2cff22 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:34.963546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c406b8-bdc5-4278-a1de-d13aba2cff22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:34.973450 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c406b8-bdc5-4278-a1de-d13aba2cff22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-94c406b8-bdc5-4278-a1de-d13aba2cff22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:39:34.978414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:39:34.979535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:39:34.980626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:39:34.980626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:39:34.980626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:39:34.980626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:39:34.980626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:39:34.980626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:39:34.980626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:39:34.980626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:39:34.980626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:39:34.980626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:39:34.980626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:39:34.980626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:39:34.980626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 592/1195] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:39:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:39:34.985767 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a98ff4-5d53-4731-936e-847b4ec93a2b None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:35.083378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a98ff4-5d53-4731-936e-847b4ec93a2b None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:35.116170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4f770-c140-4024-803e-f7a17c1eea12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:35.116821 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4f770-c140-4024-803e-f7a17c1eea12 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:35.117138 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4f770-c140-4024-803e-f7a17c1eea12 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:35.117375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4f770-c140-4024-803e-f7a17c1eea12 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:35.118085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4f770-c140-4024-803e-f7a17c1eea12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:39:35.118387 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4f770-c140-4024-803e-f7a17c1eea12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:35.119751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4f770-c140-4024-803e-f7a17c1eea12 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:35.127787 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 604/1196] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:39:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:35.134688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c406b8-bdc5-4278-a1de-d13aba2cff22 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:35.153124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:35.234546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c406b8-bdc5-4278-a1de-d13aba2cff22 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:35.269109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f990f6c-db25-4fe0-a574-0a2dd25e36c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:35.269773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f990f6c-db25-4fe0-a574-0a2dd25e36c6 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:35.270058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f990f6c-db25-4fe0-a574-0a2dd25e36c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:35.270392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f990f6c-db25-4fe0-a574-0a2dd25e36c6 None admin] PATH_INFO: `/v3/users/4c7c58a2db2d4c72a7b39828c7171774` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:35.271659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f990f6c-db25-4fe0-a574-0a2dd25e36c6 None admin] RBAC: Authorizing `identity:get_user(user_id=4c7c58a2db2d4c72a7b39828c7171774)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:35.273111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f990f6c-db25-4fe0-a574-0a2dd25e36c6 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:35.274711 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 593/1197] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Aug 3 18:39:35 2022] GET /identity/v3/users/4c7c58a2db2d4c72a7b39828c7171774 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:35.280596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4f770-c140-4024-803e-f7a17c1eea12 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:35.336442 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:35.380848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4f770-c140-4024-803e-f7a17c1eea12 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:35.413237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4defd8d-3d91-4596-9d00-3d14f769f01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:35.413843 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4defd8d-3d91-4596-9d00-3d14f769f01d None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:35.414120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4defd8d-3d91-4596-9d00-3d14f769f01d None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:35.414389 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4defd8d-3d91-4596-9d00-3d14f769f01d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:35.483825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4defd8d-3d91-4596-9d00-3d14f769f01d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:35.485367 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4defd8d-3d91-4596-9d00-3d14f769f01d None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:35.540571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:35.592204 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 605/1198] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:39:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:39:35.598453 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f990f6c-db25-4fe0-a574-0a2dd25e36c6 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:35.680869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:35.696986 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f990f6c-db25-4fe0-a574-0a2dd25e36c6 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:35.729574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9db4ea-b24f-486d-8596-6aed6e18cfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:35.730210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9db4ea-b24f-486d-8596-6aed6e18cfc5 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:35.730492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9db4ea-b24f-486d-8596-6aed6e18cfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:35.730765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9db4ea-b24f-486d-8596-6aed6e18cfc5 None admin] PATH_INFO: `/v3/projects/1f0bc8792f574431a0dd1dc519ec89e2` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:35.731911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9db4ea-b24f-486d-8596-6aed6e18cfc5 None admin] RBAC: Authorizing `identity:get_project(project_id=1f0bc8792f574431a0dd1dc519ec89e2)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:35.733369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9db4ea-b24f-486d-8596-6aed6e18cfc5 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:35.735026 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 594/1199] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Aug 3 18:39:35 2022] GET /identity/v3/projects/1f0bc8792f574431a0dd1dc519ec89e2 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:35.741507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4defd8d-3d91-4596-9d00-3d14f769f01d None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:39:35.839191 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4defd8d-3d91-4596-9d00-3d14f769f01d None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:39:35.872970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329c63c-70f1-46cc-b6ee-5c86507ec8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:39:35.873583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329c63c-70f1-46cc-b6ee-5c86507ec8a2 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:39:35.873867 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329c63c-70f1-46cc-b6ee-5c86507ec8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:39:35.874141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329c63c-70f1-46cc-b6ee-5c86507ec8a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:39:35.874933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329c63c-70f1-46cc-b6ee-5c86507ec8a2 None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=1f0bc8792f574431a0dd1dc519ec89e2, user.id=4c7c58a2db2d4c72a7b39828c7171774) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:39:35.875246 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329c63c-70f1-46cc-b6ee-5c86507ec8a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:39:35.876963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329c63c-70f1-46cc-b6ee-5c86507ec8a2 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:39:35.883449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 606/1200] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:39:35 2022] GET /identity/v3/role_assignments?user.id=4c7c58a2db2d4c72a7b39828c7171774&scope.project.id=1f0bc8792f574431a0dd1dc519ec89e2&role.id=19557e941d6f475ba92cf862846f9872 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:39:45.725510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=24 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:39:45.880841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=90 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:31.532394 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271771a0-05e9-44a0-aabf-9ae0146c2d71 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:31.533714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271771a0-05e9-44a0-aabf-9ae0146c2d71 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:31.533714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271771a0-05e9-44a0-aabf-9ae0146c2d71 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:31.535372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 595/1201] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:40:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:40:31.541337 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c4f24-30ad-4366-b8dd-97ab478ca89a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:31.541869 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c4f24-30ad-4366-b8dd-97ab478ca89a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:31.541869 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c4f24-30ad-4366-b8dd-97ab478ca89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:31.564736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-6a5c4f24-30ad-4366-b8dd-97ab478ca89a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:40:31.566255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-6a5c4f24-30ad-4366-b8dd-97ab478ca89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:40:31.659974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 607/1202] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:40:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:40:31.686483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271771a0-05e9-44a0-aabf-9ae0146c2d71 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:31.824704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271771a0-05e9-44a0-aabf-9ae0146c2d71 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:31.872275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543d56e-6884-4933-b489-10421da6dd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:31.872925 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543d56e-6884-4933-b489-10421da6dd5b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:31.873207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543d56e-6884-4933-b489-10421da6dd5b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:31.873492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543d56e-6884-4933-b489-10421da6dd5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:31.887546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543d56e-6884-4933-b489-10421da6dd5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:31.889050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543d56e-6884-4933-b489-10421da6dd5b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:31.905418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-d543d56e-6884-4933-b489-10421da6dd5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:31.905418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:31.905418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:31.905418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:31.905418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:31.905418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:31.905418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:31.905418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:31.905418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:31.905418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:31.905418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:31.905418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:31.905418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:40:31.905418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:40:31.905418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:31.906464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:31.909499 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:40:31.909499 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:40:31.909499 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:31.909499 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:31.909499 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 596/1203] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:31.917569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c4f24-30ad-4366-b8dd-97ab478ca89a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:32.052185 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c4f24-30ad-4366-b8dd-97ab478ca89a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:32.092035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:32.101084 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0aba95-3c2d-484f-bc29-5a41953a425f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:32.101709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0aba95-3c2d-484f-bc29-5a41953a425f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:32.101972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0aba95-3c2d-484f-bc29-5a41953a425f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:32.102250 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0aba95-3c2d-484f-bc29-5a41953a425f None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:32.102954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0aba95-3c2d-484f-bc29-5a41953a425f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:32.103285 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0aba95-3c2d-484f-bc29-5a41953a425f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:32.104672 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0aba95-3c2d-484f-bc29-5a41953a425f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:32.121618 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 608/1204] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:32.127692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543d56e-6884-4933-b489-10421da6dd5b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:32.198877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:32.265277 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543d56e-6884-4933-b489-10421da6dd5b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:32.316358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa51a2-9704-4434-9f30-f819a983da7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:32.317064 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa51a2-9704-4434-9f30-f819a983da7f None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:32.317417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa51a2-9704-4434-9f30-f819a983da7f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:32.317758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa51a2-9704-4434-9f30-f819a983da7f None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:32.319478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa51a2-9704-4434-9f30-f819a983da7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:32.320548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa51a2-9704-4434-9f30-f819a983da7f None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:32.360405 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 597/1205] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Aug 3 18:40:32 2022] POST /identity/v3/users => generated 251 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:40:34.681911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809764fb-8e9e-44d5-b5ff-ed3239a006e9 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:34.682521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809764fb-8e9e-44d5-b5ff-ed3239a006e9 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:34.682521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809764fb-8e9e-44d5-b5ff-ed3239a006e9 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:34.683975 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 609/1206] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:40:34.690902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a369a-2a3d-41be-b0f3-b4e90e59320a None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:34.691399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a369a-2a3d-41be-b0f3-b4e90e59320a None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:34.691478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a369a-2a3d-41be-b0f3-b4e90e59320a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:34.714220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-c01a369a-2a3d-41be-b0f3-b4e90e59320a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:40:34.715455 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c01a369a-2a3d-41be-b0f3-b4e90e59320a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:40:34.805985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 598/1207] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:40:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 18:40:34.830376 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809764fb-8e9e-44d5-b5ff-ed3239a006e9 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:34.959227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809764fb-8e9e-44d5-b5ff-ed3239a006e9 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:34.999335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa6887-3a04-47df-ad6d-a8adbeb54f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:35.000024 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa6887-3a04-47df-ad6d-a8adbeb54f48 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:35.000754 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa6887-3a04-47df-ad6d-a8adbeb54f48 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:35.001203 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa6887-3a04-47df-ad6d-a8adbeb54f48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:35.015227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa6887-3a04-47df-ad6d-a8adbeb54f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-cbaa6887-3a04-47df-ad6d-a8adbeb54f48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:40:35.021357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:35.023408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:35.025427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:35.025427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:35.025427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:35.025427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:35.025427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:40:35.025427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:40:35.025427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:40:35.025427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:40:35.025427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:40:35.025427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:40:35.025427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:40:35.025427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:40:35.025427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 610/1208] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:40:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:35.028628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a369a-2a3d-41be-b0f3-b4e90e59320a None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:35.075956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:35.150593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a369a-2a3d-41be-b0f3-b4e90e59320a None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:35.189587 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f07b6-b184-4aa4-aee1-06d329b4c259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:35.189937 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f07b6-b184-4aa4-aee1-06d329b4c259 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:35.190196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f07b6-b184-4aa4-aee1-06d329b4c259 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:35.190475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f07b6-b184-4aa4-aee1-06d329b4c259 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:35.191216 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f07b6-b184-4aa4-aee1-06d329b4c259 None admin] RBAC: Adding query filter params (name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:35.191569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f07b6-b184-4aa4-aee1-06d329b4c259 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:35.193055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f07b6-b184-4aa4-aee1-06d329b4c259 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:35.202448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 599/1209] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:40:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:40:35.209582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa6887-3a04-47df-ad6d-a8adbeb54f48 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:35.279573 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:35.343462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa6887-3a04-47df-ad6d-a8adbeb54f48 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:35.388731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245cd2d-e706-4631-9fb7-cff6fee1d5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:35.389392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245cd2d-e706-4631-9fb7-cff6fee1d5da None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:35.389680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245cd2d-e706-4631-9fb7-cff6fee1d5da None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:35.389958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245cd2d-e706-4631-9fb7-cff6fee1d5da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:35.402658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245cd2d-e706-4631-9fb7-cff6fee1d5da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:35.404342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245cd2d-e706-4631-9fb7-cff6fee1d5da None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:35.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-c245cd2d-e706-4631-9fb7-cff6fee1d5da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:35.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:35.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:35.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:35.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:35.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:35.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:35.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:35.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:35.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:35.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:35.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:35.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:40:35.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:40:35.423382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:35.424640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:35.425858 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:40:35.425858 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:40:35.425858 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:35.425858 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:40:35.425858 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 611/1210] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:35.431458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f07b6-b184-4aa4-aee1-06d329b4c259 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:35.566619 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:35.570992 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f07b6-b184-4aa4-aee1-06d329b4c259 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:35.618320 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e99076-2276-4a0d-91a7-0300e62b013c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:35.619102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e99076-2276-4a0d-91a7-0300e62b013c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:35.619456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e99076-2276-4a0d-91a7-0300e62b013c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:35.620593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e99076-2276-4a0d-91a7-0300e62b013c None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:35.621363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e99076-2276-4a0d-91a7-0300e62b013c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:35.621674 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e99076-2276-4a0d-91a7-0300e62b013c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:35.623059 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e99076-2276-4a0d-91a7-0300e62b013c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:35.638551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 600/1211] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:35.646200 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245cd2d-e706-4631-9fb7-cff6fee1d5da None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:35.765087 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245cd2d-e706-4631-9fb7-cff6fee1d5da None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:35.767779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:35.809298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86543c-0706-43c0-8149-07b419674b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:35.809953 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86543c-0706-43c0-8149-07b419674b32 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:35.810235 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86543c-0706-43c0-8149-07b419674b32 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:35.810506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86543c-0706-43c0-8149-07b419674b32 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:35.820176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86543c-0706-43c0-8149-07b419674b32 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:35.821619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86543c-0706-43c0-8149-07b419674b32 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-da86543c-0706-43c0-8149-07b419674b32 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:40:35.834728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:35.836637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:35.837606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:35.837606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:35.837606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:40:35.837606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:40:35.837606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:40:35.837606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:40:35.837606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:40:35.837606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:40:35.837606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 18:40:35.837606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:40:35.837606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 612/1212] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:35.842131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e99076-2276-4a0d-91a7-0300e62b013c None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:35.958705 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e99076-2276-4a0d-91a7-0300e62b013c None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:36.004091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a566c3-284e-445f-8eb5-4db3adbedc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:36.004777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a566c3-284e-445f-8eb5-4db3adbedc47 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:36.005058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a566c3-284e-445f-8eb5-4db3adbedc47 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:36.005355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a566c3-284e-445f-8eb5-4db3adbedc47 None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:36.006148 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a566c3-284e-445f-8eb5-4db3adbedc47 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:36.006463 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a566c3-284e-445f-8eb5-4db3adbedc47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:36.007885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a566c3-284e-445f-8eb5-4db3adbedc47 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:36.031553 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 601/1213] 10.222.0.69 () {56 vars in 1143 bytes} [Wed Aug 3 18:40:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:36.038016 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86543c-0706-43c0-8149-07b419674b32 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:36.163351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86543c-0706-43c0-8149-07b419674b32 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:36.211272 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee610186-e0d2-485f-9285-2a80c70d2ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:36.212105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee610186-e0d2-485f-9285-2a80c70d2ef1 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:36.212522 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee610186-e0d2-485f-9285-2a80c70d2ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:36.212961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee610186-e0d2-485f-9285-2a80c70d2ef1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:36.253153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:36.300671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee610186-e0d2-485f-9285-2a80c70d2ef1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:36.302129 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee610186-e0d2-485f-9285-2a80c70d2ef1 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:36.416575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 613/1214] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:40:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 18:40:36.422492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a566c3-284e-445f-8eb5-4db3adbedc47 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:36.445396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:36.556886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a566c3-284e-445f-8eb5-4db3adbedc47 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:36.603345 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905b00d-ca75-4b2e-b935-b26898ad9238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:36.604488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905b00d-ca75-4b2e-b935-b26898ad9238 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:36.604488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905b00d-ca75-4b2e-b935-b26898ad9238 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:36.604703 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905b00d-ca75-4b2e-b935-b26898ad9238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:36.618977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905b00d-ca75-4b2e-b935-b26898ad9238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:36.620809 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905b00d-ca75-4b2e-b935-b26898ad9238 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:36.637880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-6905b00d-ca75-4b2e-b935-b26898ad9238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:36.637880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:36.637880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:36.637880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:36.637880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:36.637880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:36.637880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:36.637880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:36.637880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:36.637880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:36.637880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:36.637880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:36.637880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:40:36.637880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:40:36.637880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:36.638863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:36.645692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:40:36.645692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:40:36.645692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:36.645692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:36.645692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 602/1215] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:40:36.646341 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee610186-e0d2-485f-9285-2a80c70d2ef1 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:36.716485 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:36.762596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee610186-e0d2-485f-9285-2a80c70d2ef1 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:36.805194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae80411-d51e-460f-ab4a-694a62e34a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:36.805898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae80411-d51e-460f-ab4a-694a62e34a37 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:36.806184 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae80411-d51e-460f-ab4a-694a62e34a37 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:36.806489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae80411-d51e-460f-ab4a-694a62e34a37 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:36.807173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae80411-d51e-460f-ab4a-694a62e34a37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:36.807495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae80411-d51e-460f-ab4a-694a62e34a37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:36.808959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae80411-d51e-460f-ab4a-694a62e34a37 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:36.823559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 614/1216] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:36.829771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905b00d-ca75-4b2e-b935-b26898ad9238 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:36.918766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:36.960782 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905b00d-ca75-4b2e-b935-b26898ad9238 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:37.006964 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41135d58-5f8d-4e43-a660-73088658da7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:37.007237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41135d58-5f8d-4e43-a660-73088658da7c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:37.007589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41135d58-5f8d-4e43-a660-73088658da7c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:37.007706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41135d58-5f8d-4e43-a660-73088658da7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:37.016203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41135d58-5f8d-4e43-a660-73088658da7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:37.017926 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41135d58-5f8d-4e43-a660-73088658da7c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:37.030454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-41135d58-5f8d-4e43-a660-73088658da7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:40:37.030454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:37.030454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:37.030454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:37.030454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:37.030454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:37.030454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:37.030454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:37.030454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:37.030454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:37.030454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:37.030454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:37.030454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:40:37.030454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:40:37.030454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:37.032109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:37.033595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:40:37.033595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:40:37.033595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:40:37.033595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:40:37.033595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:40:37.033595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:40:37.033595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:40:37.033595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:37.033595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 603/1217] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:40:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:37.039022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae80411-d51e-460f-ab4a-694a62e34a37 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:37.152598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae80411-d51e-460f-ab4a-694a62e34a37 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:37.178889 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:37.193735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb386d-51e1-42fe-aa38-c66924b1bb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:37.194428 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb386d-51e1-42fe-aa38-c66924b1bb7b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:37.194683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb386d-51e1-42fe-aa38-c66924b1bb7b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:37.194966 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb386d-51e1-42fe-aa38-c66924b1bb7b None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:37.195815 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb386d-51e1-42fe-aa38-c66924b1bb7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:37.196215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb386d-51e1-42fe-aa38-c66924b1bb7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:37.197775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb386d-51e1-42fe-aa38-c66924b1bb7b None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:37.211425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 615/1218] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:40:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:37.217949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41135d58-5f8d-4e43-a660-73088658da7c None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:37.335771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41135d58-5f8d-4e43-a660-73088658da7c None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:37.368143 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:37.374825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b43b3-6f78-4915-acdb-13714c852de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:37.375900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b43b3-6f78-4915-acdb-13714c852de4 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:37.376358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b43b3-6f78-4915-acdb-13714c852de4 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:37.377109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b43b3-6f78-4915-acdb-13714c852de4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:37.377928 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b43b3-6f78-4915-acdb-13714c852de4 None admin] RBAC: Adding query filter params (role.id=d67cdb8bae3648fd8f3589ab33176d14, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=fea93ceb4cbc49c2b4b280a4ddc257b7) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:37.378338 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b43b3-6f78-4915-acdb-13714c852de4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:37.379751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b43b3-6f78-4915-acdb-13714c852de4 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:37.386209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 604/1219] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:40:37 2022] GET /identity/v3/role_assignments?user.id=fea93ceb4cbc49c2b4b280a4ddc257b7&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=d67cdb8bae3648fd8f3589ab33176d14 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:39.445530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174bf9c-ffbc-456d-b6e8-942d089eeff4 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:39.446081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174bf9c-ffbc-456d-b6e8-942d089eeff4 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:39.446178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174bf9c-ffbc-456d-b6e8-942d089eeff4 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:39.447700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 616/1220] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:40:39.453474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2b0dd-3ec3-4447-8d9e-ea80c1a2f442 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:39.453907 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2b0dd-3ec3-4447-8d9e-ea80c1a2f442 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:39.454170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2b0dd-3ec3-4447-8d9e-ea80c1a2f442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:39.474060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-45b2b0dd-3ec3-4447-8d9e-ea80c1a2f442 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:40:39.475393 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-45b2b0dd-3ec3-4447-8d9e-ea80c1a2f442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:40:39.556648 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 605/1221] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:40:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:40:39.589153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174bf9c-ffbc-456d-b6e8-942d089eeff4 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:39.707361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174bf9c-ffbc-456d-b6e8-942d089eeff4 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:39.747427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabdbd5-93dc-4500-a5f0-cf13f5f1bfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:39.748081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabdbd5-93dc-4500-a5f0-cf13f5f1bfc5 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:39.748398 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabdbd5-93dc-4500-a5f0-cf13f5f1bfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:39.748677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabdbd5-93dc-4500-a5f0-cf13f5f1bfc5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:39.761010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabdbd5-93dc-4500-a5f0-cf13f5f1bfc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-2fabdbd5-93dc-4500-a5f0-cf13f5f1bfc5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:40:39.766292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:39.767658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:39.768612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:39.768612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:39.768612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:39.768612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:39.768612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:40:39.768612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:40:39.768612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:40:39.768612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:40:39.768612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:40:39.768612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:40:39.768612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:40:39.768612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:40:39.768612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 617/1222] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:40:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:40:39.773577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2b0dd-3ec3-4447-8d9e-ea80c1a2f442 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:39.893270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2b0dd-3ec3-4447-8d9e-ea80c1a2f442 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:39.920197 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:39.937941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7055ab7-c722-4e85-bb19-7e630c257a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:39.939241 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7055ab7-c722-4e85-bb19-7e630c257a77 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:39.939932 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7055ab7-c722-4e85-bb19-7e630c257a77 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:39.939932 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7055ab7-c722-4e85-bb19-7e630c257a77 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:39.940634 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7055ab7-c722-4e85-bb19-7e630c257a77 None admin] RBAC: Adding query filter params (name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:39.940952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7055ab7-c722-4e85-bb19-7e630c257a77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:39.942556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7055ab7-c722-4e85-bb19-7e630c257a77 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:39.952068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 606/1223] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:40:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:39.958350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabdbd5-93dc-4500-a5f0-cf13f5f1bfc5 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:40.007242 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:40.074504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabdbd5-93dc-4500-a5f0-cf13f5f1bfc5 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:40.117298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc6796-556e-4d2d-be8b-bd06e284461e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:40.118027 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc6796-556e-4d2d-be8b-bd06e284461e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:40.118357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc6796-556e-4d2d-be8b-bd06e284461e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:40.118663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc6796-556e-4d2d-be8b-bd06e284461e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:40.129842 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc6796-556e-4d2d-be8b-bd06e284461e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:40.131308 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc6796-556e-4d2d-be8b-bd06e284461e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:40.146748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-22bc6796-556e-4d2d-be8b-bd06e284461e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:40.146748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:40.146748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:40.146748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:40.146748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:40.146748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:40.146748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:40.146748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:40.146748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:40.146748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:40.146748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:40.146748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:40.146748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:40:40.146748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:40:40.146748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:40.147802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:40.148803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:40:40.148803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:40:40.148803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:40.148803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:40:40.148803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 618/1224] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:40.153768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7055ab7-c722-4e85-bb19-7e630c257a77 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:40.262621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7055ab7-c722-4e85-bb19-7e630c257a77 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:40.297469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483650f0-c111-47ed-9a57-257ae07fde87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:40.298672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483650f0-c111-47ed-9a57-257ae07fde87 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:40.298953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483650f0-c111-47ed-9a57-257ae07fde87 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:40.299229 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483650f0-c111-47ed-9a57-257ae07fde87 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:40.299921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483650f0-c111-47ed-9a57-257ae07fde87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:40.300213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483650f0-c111-47ed-9a57-257ae07fde87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:40.301645 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483650f0-c111-47ed-9a57-257ae07fde87 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:40.313744 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 607/1225] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:40.319581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc6796-556e-4d2d-be8b-bd06e284461e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:40.430383 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:40.437791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc6796-556e-4d2d-be8b-bd06e284461e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:40.476246 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bd72b-5b6e-4ac2-9655-e54037a7a768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:40.477093 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bd72b-5b6e-4ac2-9655-e54037a7a768 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:40.477399 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bd72b-5b6e-4ac2-9655-e54037a7a768 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:40.477794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bd72b-5b6e-4ac2-9655-e54037a7a768 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:40.486408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662bd72b-5b6e-4ac2-9655-e54037a7a768 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:40.488006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662bd72b-5b6e-4ac2-9655-e54037a7a768 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-662bd72b-5b6e-4ac2-9655-e54037a7a768 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:40:40.502898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:40.503983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:40.505299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:40.505299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:40.505299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:40:40.505299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:40:40.505299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:40:40.505299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:40:40.505299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:40:40.505299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:40:40.505299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 18:40:40.505299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:40:40.505299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 619/1226] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:40.511489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483650f0-c111-47ed-9a57-257ae07fde87 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:40.564944 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:40.621793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483650f0-c111-47ed-9a57-257ae07fde87 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:40.660598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd640e-a7d1-4e3b-8f4c-76a414cc56f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:40.661613 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd640e-a7d1-4e3b-8f4c-76a414cc56f2 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:40.661941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd640e-a7d1-4e3b-8f4c-76a414cc56f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:40.662253 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd640e-a7d1-4e3b-8f4c-76a414cc56f2 None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:40.663210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd640e-a7d1-4e3b-8f4c-76a414cc56f2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:40.663580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd640e-a7d1-4e3b-8f4c-76a414cc56f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:40.665244 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd640e-a7d1-4e3b-8f4c-76a414cc56f2 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:40.684841 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 608/1227] 10.222.0.69 () {56 vars in 1143 bytes} [Wed Aug 3 18:40:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:40:40.690753 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bd72b-5b6e-4ac2-9655-e54037a7a768 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:40.803644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bd72b-5b6e-4ac2-9655-e54037a7a768 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:40.838953 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddda56-f68c-4c97-a828-8540e14b172b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:40.839521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddda56-f68c-4c97-a828-8540e14b172b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:40.839831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddda56-f68c-4c97-a828-8540e14b172b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:40.840105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddda56-f68c-4c97-a828-8540e14b172b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:40.850903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddda56-f68c-4c97-a828-8540e14b172b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:40.852404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddda56-f68c-4c97-a828-8540e14b172b None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:40.867702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-7cddda56-f68c-4c97-a828-8540e14b172b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:40.867702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:40.867702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:40.867702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:40.867702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:40.867702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:40.867702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:40.867702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:40.867702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:40.867702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:40.867702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:40.867702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:40.867702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:40:40.867702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:40:40.867702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:40.868855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:40.869886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:40:40.869886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:40:40.869886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:40.869886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:40:40.873694 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 620/1228] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:40:40.874872 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd640e-a7d1-4e3b-8f4c-76a414cc56f2 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:40.990811 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd640e-a7d1-4e3b-8f4c-76a414cc56f2 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:40.993198 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:41.030582 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86b5d2-ba9a-4334-83bb-7be63e1f084b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:41.031209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86b5d2-ba9a-4334-83bb-7be63e1f084b None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:41.031493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86b5d2-ba9a-4334-83bb-7be63e1f084b None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:41.031768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86b5d2-ba9a-4334-83bb-7be63e1f084b None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:41.032771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86b5d2-ba9a-4334-83bb-7be63e1f084b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:41.033081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86b5d2-ba9a-4334-83bb-7be63e1f084b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:41.034458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86b5d2-ba9a-4334-83bb-7be63e1f084b None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:41.047048 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 609/1229] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:41.053746 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddda56-f68c-4c97-a828-8540e14b172b None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:41.063111 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:41.171387 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddda56-f68c-4c97-a828-8540e14b172b None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:41.209506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aaae8b-f675-4f80-8937-ad67f0bc86f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:41.210157 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aaae8b-f675-4f80-8937-ad67f0bc86f7 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:41.210444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aaae8b-f675-4f80-8937-ad67f0bc86f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:41.210718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aaae8b-f675-4f80-8937-ad67f0bc86f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:41.218802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aaae8b-f675-4f80-8937-ad67f0bc86f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:41.220869 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aaae8b-f675-4f80-8937-ad67f0bc86f7 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:41.235070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-53aaae8b-f675-4f80-8937-ad67f0bc86f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:40:41.235070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:41.235070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:41.235070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:41.235070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:41.235070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:41.235070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:41.235070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:41.235070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:41.235070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:41.235070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:41.235070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:41.235070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:40:41.235070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:40:41.235070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:41.237067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:41.238085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:40:41.238085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:40:41.238085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:40:41.238085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:40:41.238085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:40:41.238085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:40:41.238085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:40:41.238085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:40:41.238085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 621/1230] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:40:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:40:41.243223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86b5d2-ba9a-4334-83bb-7be63e1f084b None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:41.356837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86b5d2-ba9a-4334-83bb-7be63e1f084b None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:41.396274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4145a4-a546-40f9-a09a-1c3c9e4dcff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:41.396953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4145a4-a546-40f9-a09a-1c3c9e4dcff7 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:41.397319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4145a4-a546-40f9-a09a-1c3c9e4dcff7 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:41.397573 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4145a4-a546-40f9-a09a-1c3c9e4dcff7 None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:41.398411 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4145a4-a546-40f9-a09a-1c3c9e4dcff7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:41.398745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4145a4-a546-40f9-a09a-1c3c9e4dcff7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:41.400393 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4145a4-a546-40f9-a09a-1c3c9e4dcff7 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:41.413875 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 610/1231] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:40:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:40:41.421385 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aaae8b-f675-4f80-8937-ad67f0bc86f7 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:41.491913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:41.537071 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aaae8b-f675-4f80-8937-ad67f0bc86f7 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:41.580758 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cd52a-af79-4cdc-9ad6-cf8b9e16e8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:41.581443 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cd52a-af79-4cdc-9ad6-cf8b9e16e8c7 None admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:41.581720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cd52a-af79-4cdc-9ad6-cf8b9e16e8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:41.582023 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cd52a-af79-4cdc-9ad6-cf8b9e16e8c7 None admin] PATH_INFO: `/v3/projects/59f00389bb404b4392d98e22e045c84f/users/fea93ceb4cbc49c2b4b280a4ddc257b7/roles/d67cdb8bae3648fd8f3589ab33176d14` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:41.598617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cd52a-af79-4cdc-9ad6-cf8b9e16e8c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=59f00389bb404b4392d98e22e045c84f, user_id=fea93ceb4cbc49c2b4b280a4ddc257b7, role_id=d67cdb8bae3648fd8f3589ab33176d14)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:41.600086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cd52a-af79-4cdc-9ad6-cf8b9e16e8c7 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:41.614764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 622/1232] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:40:41 2022] PUT /identity/v3/projects/59f00389bb404b4392d98e22e045c84f/users/fea93ceb4cbc49c2b4b280a4ddc257b7/roles/d67cdb8bae3648fd8f3589ab33176d14 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:40:43.847204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3580784-4213-46c6-8767-f0583c18c0e3 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:43.847769 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3580784-4213-46c6-8767-f0583c18c0e3 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:43.847769 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3580784-4213-46c6-8767-f0583c18c0e3 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:43.849157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 611/1233] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:40:43.854813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff03b2-0fa3-46e2-9edc-4b95d96489d8 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:43.855260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff03b2-0fa3-46e2-9edc-4b95d96489d8 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:43.855370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff03b2-0fa3-46e2-9edc-4b95d96489d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:43.876725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-eaff03b2-0fa3-46e2-9edc-4b95d96489d8 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:40:43.878088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-eaff03b2-0fa3-46e2-9edc-4b95d96489d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:40:43.960736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 623/1234] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:40:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:40:43.985724 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3580784-4213-46c6-8767-f0583c18c0e3 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:44.019903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:44.117248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3580784-4213-46c6-8767-f0583c18c0e3 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:44.153285 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546756ce-cd35-458b-acad-411a6571aab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:44.153932 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546756ce-cd35-458b-acad-411a6571aab0 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:44.154209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546756ce-cd35-458b-acad-411a6571aab0 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:44.154478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546756ce-cd35-458b-acad-411a6571aab0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:44.165447 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546756ce-cd35-458b-acad-411a6571aab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-546756ce-cd35-458b-acad-411a6571aab0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:40:44.172203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:44.180731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:44.181899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:44.181899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:44.181899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:44.181899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:44.181899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:40:44.181899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:40:44.181899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:40:44.181899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:40:44.181899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:40:44.181899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:40:44.181899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 18:40:44.181899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:44.181899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 612/1235] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Aug 3 18:40:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:44.182801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff03b2-0fa3-46e2-9edc-4b95d96489d8 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:44.233194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:44.296254 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff03b2-0fa3-46e2-9edc-4b95d96489d8 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:44.339243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61a7f4-cd26-43e7-8986-641701e91739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:44.339959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61a7f4-cd26-43e7-8986-641701e91739 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:44.340487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61a7f4-cd26-43e7-8986-641701e91739 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:44.340694 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61a7f4-cd26-43e7-8986-641701e91739 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:44.341614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61a7f4-cd26-43e7-8986-641701e91739 None admin] RBAC: Adding query filter params (name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:44.341988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61a7f4-cd26-43e7-8986-641701e91739 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:44.343476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61a7f4-cd26-43e7-8986-641701e91739 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:44.351985 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 624/1236] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Aug 3 18:40:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:40:44.358218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546756ce-cd35-458b-acad-411a6571aab0 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:44.470145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:44.488474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546756ce-cd35-458b-acad-411a6571aab0 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:44.536057 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548eb7df-a956-48f1-9b9a-6f211d80f4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:44.537028 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548eb7df-a956-48f1-9b9a-6f211d80f4c8 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:44.537424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548eb7df-a956-48f1-9b9a-6f211d80f4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:44.537870 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548eb7df-a956-48f1-9b9a-6f211d80f4c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:44.552729 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548eb7df-a956-48f1-9b9a-6f211d80f4c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:44.554497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548eb7df-a956-48f1-9b9a-6f211d80f4c8 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:44.571792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-548eb7df-a956-48f1-9b9a-6f211d80f4c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:44.571792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:44.571792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:44.571792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:44.571792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:44.571792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:44.571792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:44.571792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:44.571792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:44.571792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:44.571792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:44.571792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:44.571792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:40:44.571792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:40:44.571792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:44.572861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:44.574058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:40:44.574058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:40:44.574058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:44.574058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:44.574058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 613/1237] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:44.579538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61a7f4-cd26-43e7-8986-641701e91739 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:44.682346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61a7f4-cd26-43e7-8986-641701e91739 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:44.692576 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:44.721394 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757ec35-fae2-40bb-8193-ba8df485fd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:44.721945 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757ec35-fae2-40bb-8193-ba8df485fd53 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:44.722228 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757ec35-fae2-40bb-8193-ba8df485fd53 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:44.722698 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757ec35-fae2-40bb-8193-ba8df485fd53 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:44.723558 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757ec35-fae2-40bb-8193-ba8df485fd53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:44.723870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757ec35-fae2-40bb-8193-ba8df485fd53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:44.725530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757ec35-fae2-40bb-8193-ba8df485fd53 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:44.740665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 625/1238] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:44.746426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548eb7df-a956-48f1-9b9a-6f211d80f4c8 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:44.856898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548eb7df-a956-48f1-9b9a-6f211d80f4c8 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:44.898006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb127dca-ef73-45cd-b823-b21b13a8d758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:44.898720 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb127dca-ef73-45cd-b823-b21b13a8d758 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:44.899003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb127dca-ef73-45cd-b823-b21b13a8d758 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:44.899276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb127dca-ef73-45cd-b823-b21b13a8d758 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:44.903081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:44.911252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb127dca-ef73-45cd-b823-b21b13a8d758 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:44.912867 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb127dca-ef73-45cd-b823-b21b13a8d758 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-cb127dca-ef73-45cd-b823-b21b13a8d758 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:40:44.925276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:44.926414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:44.927662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:44.927662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:44.927662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:40:44.927662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:40:44.927662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:40:44.927662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:40:44.927662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:40:44.927662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:40:44.927662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 18:40:44.927662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:44.927662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 614/1239] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:44.933695 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757ec35-fae2-40bb-8193-ba8df485fd53 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:45.052004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757ec35-fae2-40bb-8193-ba8df485fd53 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:45.087862 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b36ca-62e7-440d-8ecf-c595bae02afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:45.088528 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b36ca-62e7-440d-8ecf-c595bae02afe None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:45.088809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b36ca-62e7-440d-8ecf-c595bae02afe None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:45.089114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b36ca-62e7-440d-8ecf-c595bae02afe None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:45.089938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b36ca-62e7-440d-8ecf-c595bae02afe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:45.090250 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b36ca-62e7-440d-8ecf-c595bae02afe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:45.091664 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b36ca-62e7-440d-8ecf-c595bae02afe None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:45.109994 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 626/1240] 10.222.0.69 () {56 vars in 1143 bytes} [Wed Aug 3 18:40:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:45.116268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb127dca-ef73-45cd-b823-b21b13a8d758 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:45.228825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb127dca-ef73-45cd-b823-b21b13a8d758 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:45.270297 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626adee-bc82-418f-8885-4b75a626d8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:45.271082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626adee-bc82-418f-8885-4b75a626d8c4 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:45.271466 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626adee-bc82-418f-8885-4b75a626d8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:45.271742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626adee-bc82-418f-8885-4b75a626d8c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:45.342155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:45.350144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626adee-bc82-418f-8885-4b75a626d8c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:45.351622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626adee-bc82-418f-8885-4b75a626d8c4 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:45.477815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 615/1241] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:40:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:40:45.479106 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b36ca-62e7-440d-8ecf-c595bae02afe None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:45.511112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:45.598755 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b36ca-62e7-440d-8ecf-c595bae02afe None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:45.643996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552e8c3-ca74-4825-b235-4c828849fa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:45.644679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552e8c3-ca74-4825-b235-4c828849fa34 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:45.644962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552e8c3-ca74-4825-b235-4c828849fa34 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:45.645236 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552e8c3-ca74-4825-b235-4c828849fa34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:45.658372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552e8c3-ca74-4825-b235-4c828849fa34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:45.659992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552e8c3-ca74-4825-b235-4c828849fa34 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:45.677801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-7552e8c3-ca74-4825-b235-4c828849fa34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:45.677801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:45.677801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:45.677801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:45.677801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:45.677801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:45.677801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:45.677801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:45.677801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:45.677801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:45.677801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:45.677801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:45.677801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:40:45.677801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:40:45.677801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:45.679078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:45.680123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:40:45.680123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:40:45.680123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:45.680123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:40:45.680123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 627/1242] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:40:45.685725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626adee-bc82-418f-8885-4b75a626d8c4 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:45.798602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:45.805957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626adee-bc82-418f-8885-4b75a626d8c4 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:45.850596 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ff571-8c0e-4f35-a1c4-22477ea27d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:45.851372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ff571-8c0e-4f35-a1c4-22477ea27d05 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:45.851658 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ff571-8c0e-4f35-a1c4-22477ea27d05 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:45.851933 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ff571-8c0e-4f35-a1c4-22477ea27d05 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:45.852775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ff571-8c0e-4f35-a1c4-22477ea27d05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:45.853082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ff571-8c0e-4f35-a1c4-22477ea27d05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:45.854982 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ff571-8c0e-4f35-a1c4-22477ea27d05 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:45.868377 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 616/1243] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:45.874139 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552e8c3-ca74-4825-b235-4c828849fa34 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:45.966526 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:45.992050 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552e8c3-ca74-4825-b235-4c828849fa34 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:46.032417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45c3db-6663-4737-bf8f-d9eb1a5ee15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:46.033231 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45c3db-6663-4737-bf8f-d9eb1a5ee15e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:46.033535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45c3db-6663-4737-bf8f-d9eb1a5ee15e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:46.033872 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45c3db-6663-4737-bf8f-d9eb1a5ee15e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:46.041955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f45c3db-6663-4737-bf8f-d9eb1a5ee15e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:46.043959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f45c3db-6663-4737-bf8f-d9eb1a5ee15e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:46.059619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-0f45c3db-6663-4737-bf8f-d9eb1a5ee15e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:40:46.059619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:46.059619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:46.059619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:46.059619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:46.059619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:46.059619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:46.059619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:46.059619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:46.059619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:46.059619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:46.059619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:46.059619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:40:46.059619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:40:46.059619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:46.061264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:46.062434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:40:46.062434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:40:46.062434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:40:46.062434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:40:46.062434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:40:46.062434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:40:46.062434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:40:46.062434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:40:46.062434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 628/1244] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:40:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:46.069239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ff571-8c0e-4f35-a1c4-22477ea27d05 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:46.184205 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ff571-8c0e-4f35-a1c4-22477ea27d05 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:46.222993 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d2858-cfa0-4304-af86-ed2d828973de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:46.223710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d2858-cfa0-4304-af86-ed2d828973de None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:46.223981 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d2858-cfa0-4304-af86-ed2d828973de None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:46.224259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d2858-cfa0-4304-af86-ed2d828973de None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:46.225093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d2858-cfa0-4304-af86-ed2d828973de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:46.225407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d2858-cfa0-4304-af86-ed2d828973de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:46.226992 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d2858-cfa0-4304-af86-ed2d828973de None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:46.240459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 617/1245] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:40:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:46.246680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45c3db-6663-4737-bf8f-d9eb1a5ee15e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:46.366746 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45c3db-6663-4737-bf8f-d9eb1a5ee15e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:46.405644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9767e0-6126-4a5e-9bcd-6645f15a44cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:46.406270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9767e0-6126-4a5e-9bcd-6645f15a44cf None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:46.406551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9767e0-6126-4a5e-9bcd-6645f15a44cf None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:46.406834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9767e0-6126-4a5e-9bcd-6645f15a44cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:46.408158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9767e0-6126-4a5e-9bcd-6645f15a44cf None admin] RBAC: Adding query filter params (role.id=d67cdb8bae3648fd8f3589ab33176d14, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=fea93ceb4cbc49c2b4b280a4ddc257b7) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:46.408500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9767e0-6126-4a5e-9bcd-6645f15a44cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:46.410409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9767e0-6126-4a5e-9bcd-6645f15a44cf None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:46.417719 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 629/1246] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:40:46 2022] GET /identity/v3/role_assignments?user.id=fea93ceb4cbc49c2b4b280a4ddc257b7&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=d67cdb8bae3648fd8f3589ab33176d14 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:48.791377 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ef8ff-7e60-4684-bcfc-df0c0e06d570 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:48.793034 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ef8ff-7e60-4684-bcfc-df0c0e06d570 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:48.793034 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ef8ff-7e60-4684-bcfc-df0c0e06d570 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:48.793949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 618/1247] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:40:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:40:48.799814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be7f48-9e59-4eaa-a1be-98b95c418fcf None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:48.800275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be7f48-9e59-4eaa-a1be-98b95c418fcf None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:48.800412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be7f48-9e59-4eaa-a1be-98b95c418fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:48.807102 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:48.820481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-c1be7f48-9e59-4eaa-a1be-98b95c418fcf None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:40:48.821680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c1be7f48-9e59-4eaa-a1be-98b95c418fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:40:48.899646 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 630/1248] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:40:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:40:48.927813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ef8ff-7e60-4684-bcfc-df0c0e06d570 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:48.949326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:49.056628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ef8ff-7e60-4684-bcfc-df0c0e06d570 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:49.102063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd9ba6-0c9e-496c-b698-063b3575bee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:49.102807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd9ba6-0c9e-496c-b698-063b3575bee7 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:49.103166 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd9ba6-0c9e-496c-b698-063b3575bee7 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:49.103514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd9ba6-0c9e-496c-b698-063b3575bee7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:49.116043 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd9ba6-0c9e-496c-b698-063b3575bee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-b2bd9ba6-0c9e-496c-b698-063b3575bee7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:40:49.121723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:49.123667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:49.124793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:49.124793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:49.124793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:49.124793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:49.124793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:40:49.124793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:40:49.124793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:40:49.124793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:40:49.124793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:40:49.124793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:40:49.124793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:40:49.124793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:49.124793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 619/1249] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:40:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:49.130473 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be7f48-9e59-4eaa-a1be-98b95c418fcf None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:49.255538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be7f48-9e59-4eaa-a1be-98b95c418fcf None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:49.294821 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:49.300949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7006ee6-8f82-4341-8ca0-3d06e33d61d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:49.301797 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7006ee6-8f82-4341-8ca0-3d06e33d61d5 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:49.302119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7006ee6-8f82-4341-8ca0-3d06e33d61d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:49.302506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7006ee6-8f82-4341-8ca0-3d06e33d61d5 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:49.307294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7006ee6-8f82-4341-8ca0-3d06e33d61d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:49.307294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7006ee6-8f82-4341-8ca0-3d06e33d61d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:49.307294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7006ee6-8f82-4341-8ca0-3d06e33d61d5 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:49.322885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 631/1250] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:40:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:49.330367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd9ba6-0c9e-496c-b698-063b3575bee7 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:49.425780 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:49.453347 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd9ba6-0c9e-496c-b698-063b3575bee7 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:49.497159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd089c70-4fab-4abc-8d6d-2ba0da97b46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:49.497848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd089c70-4fab-4abc-8d6d-2ba0da97b46a None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:49.498172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd089c70-4fab-4abc-8d6d-2ba0da97b46a None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:49.498504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd089c70-4fab-4abc-8d6d-2ba0da97b46a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:49.519118 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd089c70-4fab-4abc-8d6d-2ba0da97b46a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:49.520919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd089c70-4fab-4abc-8d6d-2ba0da97b46a None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:49.540047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-fd089c70-4fab-4abc-8d6d-2ba0da97b46a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:49.540047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:49.540047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:49.540047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:49.540047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:49.540047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:49.540047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:49.540047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:49.540047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:49.540047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:49.540047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:49.540047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:49.540047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:40:49.540047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:40:49.540047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:49.545545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:49.546602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:40:49.546602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:40:49.546602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:49.546602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:49.546602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 620/1251] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:49.548548 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7006ee6-8f82-4341-8ca0-3d06e33d61d5 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:49.671757 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7006ee6-8f82-4341-8ca0-3d06e33d61d5 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:49.711019 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced6bc2-5c6b-483a-8dea-df11c1a8f7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:49.711727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced6bc2-5c6b-483a-8dea-df11c1a8f7d8 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:49.712013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced6bc2-5c6b-483a-8dea-df11c1a8f7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:49.712313 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced6bc2-5c6b-483a-8dea-df11c1a8f7d8 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:49.713020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced6bc2-5c6b-483a-8dea-df11c1a8f7d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:49.713351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced6bc2-5c6b-483a-8dea-df11c1a8f7d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:49.714703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced6bc2-5c6b-483a-8dea-df11c1a8f7d8 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:49.731256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 632/1252] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:49.737850 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd089c70-4fab-4abc-8d6d-2ba0da97b46a None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:49.859599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd089c70-4fab-4abc-8d6d-2ba0da97b46a None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:49.890938 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:49.900602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a63395-e5af-45e2-91c4-90f044994b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:49.901562 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a63395-e5af-45e2-91c4-90f044994b7c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:49.901893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a63395-e5af-45e2-91c4-90f044994b7c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:49.902313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a63395-e5af-45e2-91c4-90f044994b7c None admin] PATH_INFO: `/v3/users/placement` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:49.912627 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a63395-e5af-45e2-91c4-90f044994b7c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:49.914498 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a63395-e5af-45e2-91c4-90f044994b7c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-b3a63395-e5af-45e2-91c4-90f044994b7c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:40:49.929758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:49.930940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:49.931892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:49.931892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:49.931892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:40:49.931892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:40:49.931892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:40:49.931892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:40:49.931892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:40:49.931892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:40:49.931892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 18:40:49.931892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:49.931892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 621/1253] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:49.937056 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced6bc2-5c6b-483a-8dea-df11c1a8f7d8 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:50.002958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:50.062634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced6bc2-5c6b-483a-8dea-df11c1a8f7d8 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:50.113407 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadf4d9-e8ca-4c87-b4eb-9e3c9ff2d27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:50.114103 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadf4d9-e8ca-4c87-b4eb-9e3c9ff2d27b None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:50.114466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadf4d9-e8ca-4c87-b4eb-9e3c9ff2d27b None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:50.114956 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadf4d9-e8ca-4c87-b4eb-9e3c9ff2d27b None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:50.115848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadf4d9-e8ca-4c87-b4eb-9e3c9ff2d27b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:50.116159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadf4d9-e8ca-4c87-b4eb-9e3c9ff2d27b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:50.117935 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadf4d9-e8ca-4c87-b4eb-9e3c9ff2d27b None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:50.142013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 633/1254] 10.222.0.69 () {56 vars in 1143 bytes} [Wed Aug 3 18:40:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:50.148332 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a63395-e5af-45e2-91c4-90f044994b7c None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:50.274337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a63395-e5af-45e2-91c4-90f044994b7c None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:50.311941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87eaf2-00cc-49e9-b0c4-924c110fb9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:50.312662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87eaf2-00cc-49e9-b0c4-924c110fb9c7 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:50.312869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87eaf2-00cc-49e9-b0c4-924c110fb9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:50.313755 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87eaf2-00cc-49e9-b0c4-924c110fb9c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:50.376615 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:50.390545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87eaf2-00cc-49e9-b0c4-924c110fb9c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:50.392222 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87eaf2-00cc-49e9-b0c4-924c110fb9c7 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:50.515060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 622/1255] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:40:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:40:50.521692 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadf4d9-e8ca-4c87-b4eb-9e3c9ff2d27b None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:50.651661 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadf4d9-e8ca-4c87-b4eb-9e3c9ff2d27b None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:50.668204 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:50.692067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e574d32-7d6c-4dea-ab1f-326c84a3ea92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:50.693109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e574d32-7d6c-4dea-ab1f-326c84a3ea92 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:50.693474 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e574d32-7d6c-4dea-ab1f-326c84a3ea92 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:50.693853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e574d32-7d6c-4dea-ab1f-326c84a3ea92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:50.707632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e574d32-7d6c-4dea-ab1f-326c84a3ea92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:50.709129 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e574d32-7d6c-4dea-ab1f-326c84a3ea92 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:50.727509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-0e574d32-7d6c-4dea-ab1f-326c84a3ea92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:50.727509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:50.727509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:50.727509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:50.727509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:50.727509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:50.727509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:50.727509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:50.727509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:50.727509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:50.727509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:50.727509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:50.727509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:40:50.727509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:40:50.727509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:50.728635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:50.729751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:40:50.729751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:40:50.729751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:50.729751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:40:50.729751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 634/1256] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:50.736008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87eaf2-00cc-49e9-b0c4-924c110fb9c7 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:50.831710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:50.844857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87eaf2-00cc-49e9-b0c4-924c110fb9c7 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:50.884823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7d510-c681-4f18-b380-1c3b4b9ad797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:50.885683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7d510-c681-4f18-b380-1c3b4b9ad797 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:50.885683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7d510-c681-4f18-b380-1c3b4b9ad797 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:50.885958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7d510-c681-4f18-b380-1c3b4b9ad797 None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:50.886667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7d510-c681-4f18-b380-1c3b4b9ad797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:50.886948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7d510-c681-4f18-b380-1c3b4b9ad797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:50.888332 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7d510-c681-4f18-b380-1c3b4b9ad797 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:50.900805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 623/1257] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:40:50.907953 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e574d32-7d6c-4dea-ab1f-326c84a3ea92 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:51.034489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e574d32-7d6c-4dea-ab1f-326c84a3ea92 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:51.073239 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1cdcd3-fe4d-4ccf-aa22-6983ca5e2114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:51.073878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cdcd3-fe4d-4ccf-aa22-6983ca5e2114 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:51.074159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cdcd3-fe4d-4ccf-aa22-6983ca5e2114 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:51.074432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cdcd3-fe4d-4ccf-aa22-6983ca5e2114 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:51.081285 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1cdcd3-fe4d-4ccf-aa22-6983ca5e2114 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:51.082718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1cdcd3-fe4d-4ccf-aa22-6983ca5e2114 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:51.093879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-ac1cdcd3-fe4d-4ccf-aa22-6983ca5e2114 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:40:51.093879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:51.093879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:51.093879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:51.093879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:51.093879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:51.093879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:51.093879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:51.093879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:51.093879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:51.093879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:51.093879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:51.093879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:40:51.093879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:40:51.093879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:51.095132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:51.096245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:40:51.096245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:40:51.096245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:40:51.096245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:40:51.096245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:40:51.096245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:40:51.096245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:40:51.096245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:40:51.096245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 635/1258] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:40:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 18:40:51.101891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7d510-c681-4f18-b380-1c3b4b9ad797 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:51.231206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7d510-c681-4f18-b380-1c3b4b9ad797 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:51.273930 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d1cb3-630b-4846-9edb-adbb918d8b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:51.275131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d1cb3-630b-4846-9edb-adbb918d8b20 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:51.275507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d1cb3-630b-4846-9edb-adbb918d8b20 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:51.275709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d1cb3-630b-4846-9edb-adbb918d8b20 None admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:51.276658 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d1cb3-630b-4846-9edb-adbb918d8b20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:51.276971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d1cb3-630b-4846-9edb-adbb918d8b20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:51.278554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d1cb3-630b-4846-9edb-adbb918d8b20 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:51.292847 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 624/1259] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:40:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:51.300860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1cdcd3-fe4d-4ccf-aa22-6983ca5e2114 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:51.331265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:51.414966 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1cdcd3-fe4d-4ccf-aa22-6983ca5e2114 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:51.451307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960c4a0-2adb-47c7-a230-f985c5747066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:51.452034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960c4a0-2adb-47c7-a230-f985c5747066 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:51.452330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960c4a0-2adb-47c7-a230-f985c5747066 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:51.452634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960c4a0-2adb-47c7-a230-f985c5747066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:51.453526 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960c4a0-2adb-47c7-a230-f985c5747066 None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=fea93ceb4cbc49c2b4b280a4ddc257b7) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:51.453894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960c4a0-2adb-47c7-a230-f985c5747066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:51.455434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960c4a0-2adb-47c7-a230-f985c5747066 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:51.470598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 636/1260] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:40:51 2022] GET /identity/v3/role_assignments?user.id=fea93ceb4cbc49c2b4b280a4ddc257b7&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=19557e941d6f475ba92cf862846f9872 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:53.548911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbe27c-1bee-412f-b08c-36b37895e884 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:53.549482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbe27c-1bee-412f-b08c-36b37895e884 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:53.549482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbe27c-1bee-412f-b08c-36b37895e884 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:53.550827 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 625/1261] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:40:53.556044 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9945a-847c-4582-83bc-50b94ed4bf8e None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:53.556513 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9945a-847c-4582-83bc-50b94ed4bf8e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:53.556616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9945a-847c-4582-83bc-50b94ed4bf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:53.575503 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-26e9945a-847c-4582-83bc-50b94ed4bf8e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:40:53.576731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-26e9945a-847c-4582-83bc-50b94ed4bf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:40:53.642559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 637/1262] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:40:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:40:53.664806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbe27c-1bee-412f-b08c-36b37895e884 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:53.676134 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:53.768159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbe27c-1bee-412f-b08c-36b37895e884 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:53.801053 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29f31e-3f86-4411-9dc9-53b12b640c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:53.801620 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29f31e-3f86-4411-9dc9-53b12b640c39 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:53.801903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29f31e-3f86-4411-9dc9-53b12b640c39 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:53.802178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29f31e-3f86-4411-9dc9-53b12b640c39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:53.811995 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29f31e-3f86-4411-9dc9-53b12b640c39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-7d29f31e-3f86-4411-9dc9-53b12b640c39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:40:53.817399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:53.818549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:53.819625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:53.819625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:53.819625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:53.819625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:53.819625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:40:53.819625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:40:53.819625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:40:53.819625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:40:53.819625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:40:53.819625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:40:53.819625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:40:53.819625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:53.819625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 626/1263] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:40:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:53.824645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9945a-847c-4582-83bc-50b94ed4bf8e None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:53.843406 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:53.924742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9945a-847c-4582-83bc-50b94ed4bf8e None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:53.958039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f711a-00b0-42ef-b943-fe3d3be01cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:53.958649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f711a-00b0-42ef-b943-fe3d3be01cf1 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:53.958925 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f711a-00b0-42ef-b943-fe3d3be01cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:53.959194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f711a-00b0-42ef-b943-fe3d3be01cf1 None admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:53.959901 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f711a-00b0-42ef-b943-fe3d3be01cf1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:53.960206 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f711a-00b0-42ef-b943-fe3d3be01cf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:53.961593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f711a-00b0-42ef-b943-fe3d3be01cf1 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:53.969368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 638/1264] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:40:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:53.975239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29f31e-3f86-4411-9dc9-53b12b640c39 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:54.041712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:54.075492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29f31e-3f86-4411-9dc9-53b12b640c39 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:54.107862 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63eb672-588f-411f-8caa-174820e4db51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:54.108555 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63eb672-588f-411f-8caa-174820e4db51 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:54.108987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63eb672-588f-411f-8caa-174820e4db51 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:54.109227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63eb672-588f-411f-8caa-174820e4db51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:54.120094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63eb672-588f-411f-8caa-174820e4db51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:54.121577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63eb672-588f-411f-8caa-174820e4db51 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:54.137136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-c63eb672-588f-411f-8caa-174820e4db51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:54.137136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:54.137136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:54.137136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:54.137136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:54.137136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:54.137136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:54.137136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:54.137136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:54.137136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:54.137136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:54.137136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:54.137136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:40:54.137136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:40:54.137136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:54.138193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:54.139468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:40:54.139468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:40:54.139468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:54.139468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:54.139468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 627/1265] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:54.144276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f711a-00b0-42ef-b943-fe3d3be01cf1 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:54.224280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:54.244352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f711a-00b0-42ef-b943-fe3d3be01cf1 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:54.278035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b929c-36ec-4a2f-867f-92ab34a6d890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:54.278650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b929c-36ec-4a2f-867f-92ab34a6d890 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:54.278936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b929c-36ec-4a2f-867f-92ab34a6d890 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:54.279211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b929c-36ec-4a2f-867f-92ab34a6d890 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:54.279904 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b929c-36ec-4a2f-867f-92ab34a6d890 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:54.280209 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b929c-36ec-4a2f-867f-92ab34a6d890 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:54.282147 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b929c-36ec-4a2f-867f-92ab34a6d890 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:54.293771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 639/1266] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:54.300173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63eb672-588f-411f-8caa-174820e4db51 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:54.399427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63eb672-588f-411f-8caa-174820e4db51 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:54.419197 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:54.434224 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be9f0d-d32b-454f-a8ec-ad7c0240af8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:54.434864 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be9f0d-d32b-454f-a8ec-ad7c0240af8e None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:54.435151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be9f0d-d32b-454f-a8ec-ad7c0240af8e None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:54.435429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be9f0d-d32b-454f-a8ec-ad7c0240af8e None admin] PATH_INFO: `/v3/users/placement` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:54.443137 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be9f0d-d32b-454f-a8ec-ad7c0240af8e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:54.444619 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be9f0d-d32b-454f-a8ec-ad7c0240af8e None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-a6be9f0d-d32b-454f-a8ec-ad7c0240af8e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:40:54.455572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:54.456768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:54.457821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:54.457821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:54.457821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:40:54.457821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:40:54.457821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:40:54.457821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:40:54.457821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:40:54.457821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:40:54.457821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 18:40:54.457821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:54.457821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 628/1267] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:54.462617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b929c-36ec-4a2f-867f-92ab34a6d890 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:54.572172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b929c-36ec-4a2f-867f-92ab34a6d890 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:54.606291 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2031f-4dc9-42ea-b474-72dd0acb0dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:54.606900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2031f-4dc9-42ea-b474-72dd0acb0dd0 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:54.607176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2031f-4dc9-42ea-b474-72dd0acb0dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:54.607445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2031f-4dc9-42ea-b474-72dd0acb0dd0 None admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:54.608231 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2031f-4dc9-42ea-b474-72dd0acb0dd0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:54.608564 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2031f-4dc9-42ea-b474-72dd0acb0dd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:54.610700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2031f-4dc9-42ea-b474-72dd0acb0dd0 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:54.619073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:54.628255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 640/1268] 10.222.0.69 () {56 vars in 1143 bytes} [Wed Aug 3 18:40:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:54.634022 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be9f0d-d32b-454f-a8ec-ad7c0240af8e None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:54.732949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be9f0d-d32b-454f-a8ec-ad7c0240af8e None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:54.765676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4e19f-629f-4737-b5fc-9dbe2fe061ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:54.767088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4e19f-629f-4737-b5fc-9dbe2fe061ef None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:54.767208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4e19f-629f-4737-b5fc-9dbe2fe061ef None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:54.767476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4e19f-629f-4737-b5fc-9dbe2fe061ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:54.777705 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4e19f-629f-4737-b5fc-9dbe2fe061ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:54.779140 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4e19f-629f-4737-b5fc-9dbe2fe061ef None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:54.793575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-43e4e19f-629f-4737-b5fc-9dbe2fe061ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:54.793575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:54.793575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:54.793575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:54.793575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:54.793575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:54.793575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:54.793575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:54.793575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:54.793575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:54.793575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:54.793575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:54.793575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:40:54.793575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:40:54.793575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:54.794635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:54.795765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:40:54.795765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:40:54.795765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:54.795765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:54.795765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 629/1269] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:54.800592 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2031f-4dc9-42ea-b474-72dd0acb0dd0 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:54.898182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2031f-4dc9-42ea-b474-72dd0acb0dd0 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:54.936689 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3364-d1b9-42c6-862e-634b0eb8cf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:54.937317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3364-d1b9-42c6-862e-634b0eb8cf38 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:54.937600 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3364-d1b9-42c6-862e-634b0eb8cf38 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:54.937919 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3364-d1b9-42c6-862e-634b0eb8cf38 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:54.938741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b3364-d1b9-42c6-862e-634b0eb8cf38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:54.939301 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b3364-d1b9-42c6-862e-634b0eb8cf38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:54.940474 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b3364-d1b9-42c6-862e-634b0eb8cf38 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:54.951865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 641/1270] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:54.958489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4e19f-629f-4737-b5fc-9dbe2fe061ef None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:54.990441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:55.059356 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4e19f-629f-4737-b5fc-9dbe2fe061ef None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:55.092757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984ddb6-bf24-4bc2-aef8-72b10606cbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:55.093383 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984ddb6-bf24-4bc2-aef8-72b10606cbc5 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:55.093661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984ddb6-bf24-4bc2-aef8-72b10606cbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:55.094089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984ddb6-bf24-4bc2-aef8-72b10606cbc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:55.100751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984ddb6-bf24-4bc2-aef8-72b10606cbc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:55.102172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984ddb6-bf24-4bc2-aef8-72b10606cbc5 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:55.113228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-4984ddb6-bf24-4bc2-aef8-72b10606cbc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:40:55.113228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:55.113228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:55.113228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:55.113228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:55.113228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:55.113228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:55.113228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:55.113228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:55.113228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:55.113228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:55.113228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:55.113228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:40:55.113228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:40:55.113228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:55.114310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:55.115248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:40:55.115248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:40:55.115248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:40:55.115248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:40:55.115248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:40:55.115248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:40:55.115248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:40:55.115248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:55.115248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 630/1271] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:40:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:55.120388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3364-d1b9-42c6-862e-634b0eb8cf38 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:55.179675 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:55.222326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3364-d1b9-42c6-862e-634b0eb8cf38 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:55.257010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819785d-ceb9-4b50-af9d-3fb1abef1076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:55.257835 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819785d-ceb9-4b50-af9d-3fb1abef1076 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:55.258250 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819785d-ceb9-4b50-af9d-3fb1abef1076 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:55.258656 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819785d-ceb9-4b50-af9d-3fb1abef1076 None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:55.259813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819785d-ceb9-4b50-af9d-3fb1abef1076 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:55.260272 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819785d-ceb9-4b50-af9d-3fb1abef1076 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:55.262600 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819785d-ceb9-4b50-af9d-3fb1abef1076 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:55.280386 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 642/1272] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:40:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:55.286905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984ddb6-bf24-4bc2-aef8-72b10606cbc5 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:55.379033 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:55.388016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984ddb6-bf24-4bc2-aef8-72b10606cbc5 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:55.420145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d7c2e-121e-4a51-991d-8a97a069d1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:55.420842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d7c2e-121e-4a51-991d-8a97a069d1de None admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:55.421128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d7c2e-121e-4a51-991d-8a97a069d1de None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:55.421406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d7c2e-121e-4a51-991d-8a97a069d1de None admin] PATH_INFO: `/v3/projects/59f00389bb404b4392d98e22e045c84f/users/fea93ceb4cbc49c2b4b280a4ddc257b7/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:55.424338 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d7c2e-121e-4a51-991d-8a97a069d1de None admin] RBAC: Authorizing `identity:create_grant(project_id=59f00389bb404b4392d98e22e045c84f, user_id=fea93ceb4cbc49c2b4b280a4ddc257b7, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:55.425741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d7c2e-121e-4a51-991d-8a97a069d1de None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:55.438311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 631/1273] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Aug 3 18:40:55 2022] PUT /identity/v3/projects/59f00389bb404b4392d98e22e045c84f/users/fea93ceb4cbc49c2b4b280a4ddc257b7/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:40:57.434818 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b038e-5ed2-4688-a422-8c45fc95643d None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:57.435349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b038e-5ed2-4688-a422-8c45fc95643d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:57.435349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b038e-5ed2-4688-a422-8c45fc95643d None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:57.436771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 643/1274] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:40:57.442015 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3392e8-9b17-43d5-be02-e61f20849334 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:57.442423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3392e8-9b17-43d5-be02-e61f20849334 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:57.442575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3392e8-9b17-43d5-be02-e61f20849334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:57.460795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-dc3392e8-9b17-43d5-be02-e61f20849334 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:40:57.461975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-dc3392e8-9b17-43d5-be02-e61f20849334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:40:57.535493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 632/1275] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:40:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:40:57.559693 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b038e-5ed2-4688-a422-8c45fc95643d None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:57.659735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b038e-5ed2-4688-a422-8c45fc95643d None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:57.676710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:57.693743 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b23f8c-ef4b-4bd9-bfce-edfc55838abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:57.694309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b23f8c-ef4b-4bd9-bfce-edfc55838abf None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:57.694589 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b23f8c-ef4b-4bd9-bfce-edfc55838abf None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:57.694863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b23f8c-ef4b-4bd9-bfce-edfc55838abf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:57.705393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b23f8c-ef4b-4bd9-bfce-edfc55838abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-a7b23f8c-ef4b-4bd9-bfce-edfc55838abf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 18:40:57.710223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:57.711141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:57.712088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:57.712088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:57.712088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:57.712088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:57.712088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 18:40:57.712088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 18:40:57.712088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 18:40:57.712088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 18:40:57.712088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 18:40:57.712088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 18:40:57.712088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 18:40:57.712088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:40:57.712088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 644/1276] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Aug 3 18:40:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:57.717540 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3392e8-9b17-43d5-be02-e61f20849334 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:57.798659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:57.817473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3392e8-9b17-43d5-be02-e61f20849334 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:57.851332 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726de41-c8f3-401d-9e86-ca352ceb1780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:57.851948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726de41-c8f3-401d-9e86-ca352ceb1780 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:57.852234 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726de41-c8f3-401d-9e86-ca352ceb1780 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:57.852544 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726de41-c8f3-401d-9e86-ca352ceb1780 None admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:57.853263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726de41-c8f3-401d-9e86-ca352ceb1780 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:57.853568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726de41-c8f3-401d-9e86-ca352ceb1780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:57.854927 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726de41-c8f3-401d-9e86-ca352ceb1780 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:57.862877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 633/1277] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Aug 3 18:40:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:40:57.868547 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b23f8c-ef4b-4bd9-bfce-edfc55838abf None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:57.969502 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b23f8c-ef4b-4bd9-bfce-edfc55838abf None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:58.003158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bad8d-8b53-4090-9372-d462a4d3ebc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:58.003793 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bad8d-8b53-4090-9372-d462a4d3ebc0 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:58.004077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bad8d-8b53-4090-9372-d462a4d3ebc0 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:58.004385 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bad8d-8b53-4090-9372-d462a4d3ebc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:58.014210 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bad8d-8b53-4090-9372-d462a4d3ebc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:58.015677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bad8d-8b53-4090-9372-d462a4d3ebc0 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:58.030912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-c30bad8d-8b53-4090-9372-d462a4d3ebc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:58.030912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:58.030912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:58.030912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:58.030912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:58.030912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:58.030912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:58.030912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:58.030912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:58.030912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:58.030912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:58.030912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:58.030912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:40:58.030912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:40:58.030912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:58.032042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:58.033022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:40:58.033022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:40:58.033022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:58.033022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:40:58.033022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 645/1278] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:58.038000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726de41-c8f3-401d-9e86-ca352ceb1780 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:58.134771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726de41-c8f3-401d-9e86-ca352ceb1780 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:58.167661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac697c7b-15b2-47fd-b898-a5996cf4ff9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:58.168264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac697c7b-15b2-47fd-b898-a5996cf4ff9c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:58.168575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac697c7b-15b2-47fd-b898-a5996cf4ff9c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:58.168844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac697c7b-15b2-47fd-b898-a5996cf4ff9c None admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:58.169529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac697c7b-15b2-47fd-b898-a5996cf4ff9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:58.169825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac697c7b-15b2-47fd-b898-a5996cf4ff9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:58.171161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac697c7b-15b2-47fd-b898-a5996cf4ff9c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:58.182837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 634/1279] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:58.189082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bad8d-8b53-4090-9372-d462a4d3ebc0 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:58.219908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:58.289349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bad8d-8b53-4090-9372-d462a4d3ebc0 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:58.322647 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe3c4d-c1f1-4c48-964a-056450cfb636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:58.323279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe3c4d-c1f1-4c48-964a-056450cfb636 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:58.323560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe3c4d-c1f1-4c48-964a-056450cfb636 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:58.323834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe3c4d-c1f1-4c48-964a-056450cfb636 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:58.332760 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe3c4d-c1f1-4c48-964a-056450cfb636 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:58.334215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe3c4d-c1f1-4c48-964a-056450cfb636 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-0abe3c4d-c1f1-4c48-964a-056450cfb636 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 18:40:58.345300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:58.346404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:58.347671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:58.347671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:58.347671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 18:40:58.347671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 18:40:58.347671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 18:40:58.347671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 18:40:58.347671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 18:40:58.347671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 18:40:58.347671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 18:40:58.347671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:40:58.347671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 646/1280] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:58.352387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac697c7b-15b2-47fd-b898-a5996cf4ff9c None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:58.356103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:58.452325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac697c7b-15b2-47fd-b898-a5996cf4ff9c None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:58.487626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b5e16-74a2-4e05-be71-f311ce9ce6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:58.488173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b5e16-74a2-4e05-be71-f311ce9ce6d4 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:58.488491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b5e16-74a2-4e05-be71-f311ce9ce6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:58.488770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b5e16-74a2-4e05-be71-f311ce9ce6d4 None admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:58.489570 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b5e16-74a2-4e05-be71-f311ce9ce6d4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:58.490211 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b5e16-74a2-4e05-be71-f311ce9ce6d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:58.491342 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b5e16-74a2-4e05-be71-f311ce9ce6d4 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:58.509080 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 635/1281] 10.222.0.69 () {56 vars in 1143 bytes} [Wed Aug 3 18:40:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:58.515137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe3c4d-c1f1-4c48-964a-056450cfb636 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:58.616175 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:58.626420 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe3c4d-c1f1-4c48-964a-056450cfb636 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:58.660307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c6f83-2842-4f61-ab8e-b0368dac2f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:58.660949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c6f83-2842-4f61-ab8e-b0368dac2f9f None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:58.661228 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c6f83-2842-4f61-ab8e-b0368dac2f9f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:58.661505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c6f83-2842-4f61-ab8e-b0368dac2f9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:58.728197 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c6f83-2842-4f61-ab8e-b0368dac2f9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:58.729677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c6f83-2842-4f61-ab8e-b0368dac2f9f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:58.816425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:58.826729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 647/1282] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Aug 3 18:40:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 18:40:58.833379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b5e16-74a2-4e05-be71-f311ce9ce6d4 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:58.894000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:58.931864 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b5e16-74a2-4e05-be71-f311ce9ce6d4 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:58.966482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb1599-f0b2-4a27-a5c3-2b84f298b157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:58.967124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb1599-f0b2-4a27-a5c3-2b84f298b157 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:58.967407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb1599-f0b2-4a27-a5c3-2b84f298b157 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:58.967690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb1599-f0b2-4a27-a5c3-2b84f298b157 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:58.977906 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb1599-f0b2-4a27-a5c3-2b84f298b157 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:58.979440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb1599-f0b2-4a27-a5c3-2b84f298b157 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:58.995367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-86fb1599-f0b2-4a27-a5c3-2b84f298b157 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:58.995367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:58.995367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:58.995367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:58.995367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:58.995367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:58.995367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:58.995367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:58.995367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:58.995367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:58.995367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:58.995367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:58.995367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 18:40:58.995367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 18:40:58.995367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:58.996793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:58.997894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 18:40:58.997894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 18:40:58.997894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 18:40:58.997894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:58.997894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 636/1283] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:59.003479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c6f83-2842-4f61-ab8e-b0368dac2f9f None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:59.101125 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c6f83-2842-4f61-ab8e-b0368dac2f9f None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:59.135525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a0673-0e48-4fba-aa36-39fa16302388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:59.136143 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a0673-0e48-4fba-aa36-39fa16302388 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:59.136482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a0673-0e48-4fba-aa36-39fa16302388 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:59.136758 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a0673-0e48-4fba-aa36-39fa16302388 None admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:59.137451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a0673-0e48-4fba-aa36-39fa16302388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:59.137756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a0673-0e48-4fba-aa36-39fa16302388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:59.139151 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a0673-0e48-4fba-aa36-39fa16302388 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:59.150225 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 648/1284] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Aug 3 18:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:59.155839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb1599-f0b2-4a27-a5c3-2b84f298b157 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:59.256671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb1599-f0b2-4a27-a5c3-2b84f298b157 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:59.269979 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:59.290593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417dbb8a-58f8-45f7-9efc-8bbc5d45d899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:59.291285 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dbb8a-58f8-45f7-9efc-8bbc5d45d899 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:59.291574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dbb8a-58f8-45f7-9efc-8bbc5d45d899 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:59.291847 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dbb8a-58f8-45f7-9efc-8bbc5d45d899 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:59.299259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417dbb8a-58f8-45f7-9efc-8bbc5d45d899 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:59.300796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417dbb8a-58f8-45f7-9efc-8bbc5d45d899 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:59.311844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application [None req-417dbb8a-58f8-45f7-9efc-8bbc5d45d899 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:40:59.311844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:40:59.311844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:40:59.311844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:40:59.311844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:40:59.311844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:40:59.311844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:40:59.311844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:40:59.311844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:40:59.311844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:40:59.311844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:40:59.311844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:40:59.311844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 18:40:59.311844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 18:40:59.311844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application with Lock( Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:40:59.313683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:40:59.314646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 18:40:59.314646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 18:40:59.314646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 18:40:59.314646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 18:40:59.314646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 18:40:59.314646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 18:40:59.314646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 18:40:59.314646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: ERROR keystone.server.flask.application Aug 03 18:40:59.314646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 637/1285] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Aug 3 18:40:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:40:59.320275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a0673-0e48-4fba-aa36-39fa16302388 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:59.377917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:40:59.428455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a0673-0e48-4fba-aa36-39fa16302388 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:59.462960 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336300a-1065-400c-af7a-13ff6cfaca7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:59.463575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336300a-1065-400c-af7a-13ff6cfaca7c None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:59.463856 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336300a-1065-400c-af7a-13ff6cfaca7c None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:59.464130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336300a-1065-400c-af7a-13ff6cfaca7c None admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:59.464932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336300a-1065-400c-af7a-13ff6cfaca7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:59.465247 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336300a-1065-400c-af7a-13ff6cfaca7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:59.466676 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336300a-1065-400c-af7a-13ff6cfaca7c None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:59.478939 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 649/1286] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Aug 3 18:40:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:40:59.485502 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417dbb8a-58f8-45f7-9efc-8bbc5d45d899 None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:40:59.594054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417dbb8a-58f8-45f7-9efc-8bbc5d45d899 None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:40:59.627321 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648ea67-dcea-441e-a4cf-f4db67903875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:40:59.627992 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648ea67-dcea-441e-a4cf-f4db67903875 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:40:59.628224 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648ea67-dcea-441e-a4cf-f4db67903875 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:40:59.628532 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648ea67-dcea-441e-a4cf-f4db67903875 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:40:59.629325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e648ea67-dcea-441e-a4cf-f4db67903875 None admin] RBAC: Adding query filter params (role.id=19557e941d6f475ba92cf862846f9872, scope.project.id=59f00389bb404b4392d98e22e045c84f, user.id=fea93ceb4cbc49c2b4b280a4ddc257b7) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:40:59.629636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e648ea67-dcea-441e-a4cf-f4db67903875 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:40:59.631233 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e648ea67-dcea-441e-a4cf-f4db67903875 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:40:59.637214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 638/1287] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Aug 3 18:40:59 2022] GET /identity/v3/role_assignments?user.id=fea93ceb4cbc49c2b4b280a4ddc257b7&scope.project.id=59f00389bb404b4392d98e22e045c84f&role.id=19557e941d6f475ba92cf862846f9872 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:41:01.667543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ffe44-f055-4b60-882c-10752001e173 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:01.668111 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ffe44-f055-4b60-882c-10752001e173 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:01.668111 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ffe44-f055-4b60-882c-10752001e173 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:01.669501 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 650/1288] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:41:01.675257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6cf45-eb18-4cdc-a558-ee849cf38d92 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:01.675666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6cf45-eb18-4cdc-a558-ee849cf38d92 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:01.675811 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6cf45-eb18-4cdc-a558-ee849cf38d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:01.694068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-28e6cf45-eb18-4cdc-a558-ee849cf38d92 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:41:01.695227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-28e6cf45-eb18-4cdc-a558-ee849cf38d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:41:01.724598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:41:01.761600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 639/1289] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:41:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:41:01.784118 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ffe44-f055-4b60-882c-10752001e173 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:41:01.883079 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ffe44-f055-4b60-882c-10752001e173 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:41:01.894103 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:41:01.918108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e45230-822c-421d-9fdc-725fbc00fb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:41:01.918682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e45230-822c-421d-9fdc-725fbc00fb25 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:01.918964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e45230-822c-421d-9fdc-725fbc00fb25 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:01.919239 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e45230-822c-421d-9fdc-725fbc00fb25 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:01.919868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e45230-822c-421d-9fdc-725fbc00fb25 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:41:01.921268 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e45230-822c-421d-9fdc-725fbc00fb25 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-21e45230-822c-421d-9fdc-725fbc00fb25 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:41:01.930169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:41:01.932517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:41:01.933441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:41:01.933441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:41:01.933441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:41:01.933441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:41:01.933441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:41:01.933441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:41:01.933441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:41:01.933441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:41:01.933441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:41:01.933441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:41:01.933441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:41:01.933441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:41:01.933441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 18:41:01.933441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:41:01.933441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 651/1290] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Aug 3 18:41:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:41:01.937231 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6cf45-eb18-4cdc-a558-ee849cf38d92 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:41:02.041582 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6cf45-eb18-4cdc-a558-ee849cf38d92 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:41:02.075000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbe69d-a0ca-4e97-820a-ce1d950fc6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:41:02.075619 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbe69d-a0ca-4e97-820a-ce1d950fc6b0 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:02.075902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbe69d-a0ca-4e97-820a-ce1d950fc6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:02.076173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbe69d-a0ca-4e97-820a-ce1d950fc6b0 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:02.076882 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbe69d-a0ca-4e97-820a-ce1d950fc6b0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:41:02.077187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbe69d-a0ca-4e97-820a-ce1d950fc6b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:41:02.079387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbe69d-a0ca-4e97-820a-ce1d950fc6b0 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:41:02.084151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 640/1291] 10.222.0.69 () {56 vars in 1116 bytes} [Wed Aug 3 18:41:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:41:02.089875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e45230-822c-421d-9fdc-725fbc00fb25 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:41:02.189670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e45230-822c-421d-9fdc-725fbc00fb25 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:41:02.221939 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501f94c-8ed3-45f8-8931-5d19c999edb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:41:02.222550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501f94c-8ed3-45f8-8931-5d19c999edb6 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:02.222834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501f94c-8ed3-45f8-8931-5d19c999edb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:02.223109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501f94c-8ed3-45f8-8931-5d19c999edb6 None admin] PATH_INFO: `/v3/services` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:02.223785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501f94c-8ed3-45f8-8931-5d19c999edb6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:41:02.224090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501f94c-8ed3-45f8-8931-5d19c999edb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:41:02.225502 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501f94c-8ed3-45f8-8931-5d19c999edb6 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:41:02.229899 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 652/1292] 10.222.0.69 () {56 vars in 1116 bytes} [Wed Aug 3 18:41:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:41:04.207722 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9728d-0233-4bfd-bac8-18778b74ba5f None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:04.208264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9728d-0233-4bfd-bac8-18778b74ba5f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:04.208264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9728d-0233-4bfd-bac8-18778b74ba5f None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:04.209697 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 641/1293] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:41:04.214833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c056338-5f5c-4f5a-b026-34d280417616 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:04.215286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c056338-5f5c-4f5a-b026-34d280417616 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:04.215391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c056338-5f5c-4f5a-b026-34d280417616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:04.233832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-4c056338-5f5c-4f5a-b026-34d280417616 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:41:04.236070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4c056338-5f5c-4f5a-b026-34d280417616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:41:04.264793 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:41:04.303412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 653/1294] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:41:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:41:04.325931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9728d-0233-4bfd-bac8-18778b74ba5f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:41:04.357348 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:41:04.430454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9728d-0233-4bfd-bac8-18778b74ba5f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:41:04.463772 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992f766-bdb1-49f0-b496-0c508000787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:41:04.464342 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992f766-bdb1-49f0-b496-0c508000787f None admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:04.464627 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992f766-bdb1-49f0-b496-0c508000787f None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:04.464901 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992f766-bdb1-49f0-b496-0c508000787f None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:04.465601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992f766-bdb1-49f0-b496-0c508000787f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:41:04.466988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992f766-bdb1-49f0-b496-0c508000787f None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:41:04.481372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 642/1295] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Aug 3 18:41:04 2022] POST /identity/v3/services => generated 246 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:41:06.564841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7be7f6-32f6-4e5f-8e71-23d213e07e6b None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:06.565445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7be7f6-32f6-4e5f-8e71-23d213e07e6b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:06.565546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7be7f6-32f6-4e5f-8e71-23d213e07e6b None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:06.566879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 654/1296] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:41:06.574690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77bc01-431c-49d6-b546-831a21e67bf4 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:06.575093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77bc01-431c-49d6-b546-831a21e67bf4 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:06.575232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77bc01-431c-49d6-b546-831a21e67bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:06.594970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-bf77bc01-431c-49d6-b546-831a21e67bf4 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:41:06.596167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-bf77bc01-431c-49d6-b546-831a21e67bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:41:06.676790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 643/1297] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:41:06 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:41:06.699626 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7be7f6-32f6-4e5f-8e71-23d213e07e6b None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:41:06.807873 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7be7f6-32f6-4e5f-8e71-23d213e07e6b None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:41:06.848083 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f811924-484c-49ff-87aa-0fd96cc07c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:41:06.848678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f811924-484c-49ff-87aa-0fd96cc07c77 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:06.848964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f811924-484c-49ff-87aa-0fd96cc07c77 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:06.849240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f811924-484c-49ff-87aa-0fd96cc07c77 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:06.849881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f811924-484c-49ff-87aa-0fd96cc07c77 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:41:06.851254 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f811924-484c-49ff-87aa-0fd96cc07c77 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-1f811924-484c-49ff-87aa-0fd96cc07c77 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:41:06.866458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:41:06.867718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:41:06.868803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:41:06.868803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:41:06.868803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:41:06.868803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:41:06.868803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:41:06.868803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:41:06.868803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:41:06.868803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:41:06.868803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:41:06.868803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:41:06.868803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:41:06.868803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:41:06.868803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 18:41:06.868803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:41:06.868803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 655/1298] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Aug 3 18:41:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:41:06.873663 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77bc01-431c-49d6-b546-831a21e67bf4 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:41:06.892448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:41:06.975154 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77bc01-431c-49d6-b546-831a21e67bf4 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:41:07.008898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad75cb-8c32-4fbe-b42f-d9ee7f40518c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:41:07.009517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad75cb-8c32-4fbe-b42f-d9ee7f40518c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:07.009800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad75cb-8c32-4fbe-b42f-d9ee7f40518c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:07.010074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad75cb-8c32-4fbe-b42f-d9ee7f40518c None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:07.010783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad75cb-8c32-4fbe-b42f-d9ee7f40518c None admin] RBAC: Adding query filter params (name=placement) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:41:07.011088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad75cb-8c32-4fbe-b42f-d9ee7f40518c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:41:07.012494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad75cb-8c32-4fbe-b42f-d9ee7f40518c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:41:07.017719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 644/1299] 10.222.0.69 () {56 vars in 1116 bytes} [Wed Aug 3 18:41:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:41:07.023599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f811924-484c-49ff-87aa-0fd96cc07c77 None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:41:07.055864 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:41:07.123605 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f811924-484c-49ff-87aa-0fd96cc07c77 None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:41:07.158001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124c5ae-cdb4-4720-9e18-c65199a19449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:41:07.158616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124c5ae-cdb4-4720-9e18-c65199a19449 None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:07.158956 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124c5ae-cdb4-4720-9e18-c65199a19449 None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:07.159455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124c5ae-cdb4-4720-9e18-c65199a19449 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:07.160270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124c5ae-cdb4-4720-9e18-c65199a19449 None admin] RBAC: Adding query filter params (interface=public, service_id=e2c2d5648a0545ada828bab5bd7453be, region_id=RegionOne) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:41:07.160619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124c5ae-cdb4-4720-9e18-c65199a19449 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:41:07.161992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124c5ae-cdb4-4720-9e18-c65199a19449 None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:41:07.167005 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 656/1300] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Aug 3 18:41:07 2022] GET /identity/v3/endpoints?service_id=e2c2d5648a0545ada828bab5bd7453be&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:41:07.173476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad75cb-8c32-4fbe-b42f-d9ee7f40518c None admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:41:07.252968 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:41:07.271001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad75cb-8c32-4fbe-b42f-d9ee7f40518c None admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:41:07.304757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a93211-60a4-48ac-b2a8-c865e4044d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:41:07.305367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a93211-60a4-48ac-b2a8-c865e4044d43 None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:07.305658 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a93211-60a4-48ac-b2a8-c865e4044d43 None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:07.305933 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a93211-60a4-48ac-b2a8-c865e4044d43 None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:07.306583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a93211-60a4-48ac-b2a8-c865e4044d43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:41:07.307966 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a93211-60a4-48ac-b2a8-c865e4044d43 None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:41:07.314892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 645/1301] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Aug 3 18:41:07 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:41:09.356516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e7428-8a82-4c4d-9bd8-4fbf183b6c12 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:09.357076 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e7428-8a82-4c4d-9bd8-4fbf183b6c12 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:09.357076 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e7428-8a82-4c4d-9bd8-4fbf183b6c12 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:09.358432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 657/1302] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:41:09.363634 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c9273-654b-48b2-9d9e-094b3006de03 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:09.364058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c9273-654b-48b2-9d9e-094b3006de03 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:09.364177 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c9273-654b-48b2-9d9e-094b3006de03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:09.382517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-866c9273-654b-48b2-9d9e-094b3006de03 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:41:09.383745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-866c9273-654b-48b2-9d9e-094b3006de03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:41:09.450090 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 646/1303] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:41:09 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 18:41:09.473952 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e7428-8a82-4c4d-9bd8-4fbf183b6c12 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:41:09.568757 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:41:09.575661 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e7428-8a82-4c4d-9bd8-4fbf183b6c12 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:41:09.611031 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7925aa3-f2f9-4044-8d95-adbaa0df195e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:41:09.611597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7925aa3-f2f9-4044-8d95-adbaa0df195e None admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:09.611881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7925aa3-f2f9-4044-8d95-adbaa0df195e None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:09.612158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7925aa3-f2f9-4044-8d95-adbaa0df195e None admin] PATH_INFO: `/v3/services/placement` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:09.612818 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7925aa3-f2f9-4044-8d95-adbaa0df195e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:41:09.614187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7925aa3-f2f9-4044-8d95-adbaa0df195e None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application [None req-c7925aa3-f2f9-4044-8d95-adbaa0df195e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 18:41:09.622988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 18:41:09.624148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 18:41:09.626307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 18:41:09.626307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application with Lock( Aug 03 18:41:09.626307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 18:41:09.626307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self._enter() Aug 03 18:41:09.626307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 18:41:09.626307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 18:41:09.626307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 18:41:09.626307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application return self.creator() Aug 03 18:41:09.626307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 18:41:09.626307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application created_value = creator( Aug 03 18:41:09.626307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 18:41:09.626307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 18:41:09.626307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 18:41:09.626307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: ERROR keystone.server.flask.application Aug 03 18:41:09.626307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 658/1304] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Aug 3 18:41:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 18:41:09.631205 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c9273-654b-48b2-9d9e-094b3006de03 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:41:09.706376 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:41:09.728674 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c9273-654b-48b2-9d9e-094b3006de03 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:41:09.763062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d0721-2315-4d67-ba35-48b1f993dd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:41:09.763671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d0721-2315-4d67-ba35-48b1f993dd9c None admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:09.763949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d0721-2315-4d67-ba35-48b1f993dd9c None admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:09.764220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d0721-2315-4d67-ba35-48b1f993dd9c None admin] PATH_INFO: `/v3/services` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:09.764921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d0721-2315-4d67-ba35-48b1f993dd9c None admin] RBAC: Adding query filter params (name=placement) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:41:09.765361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d0721-2315-4d67-ba35-48b1f993dd9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:41:09.766721 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d0721-2315-4d67-ba35-48b1f993dd9c None admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:41:09.771754 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 647/1305] 10.222.0.69 () {56 vars in 1116 bytes} [Wed Aug 3 18:41:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:41:09.777543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7925aa3-f2f9-4044-8d95-adbaa0df195e None admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:41:09.879477 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7925aa3-f2f9-4044-8d95-adbaa0df195e None admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:41:09.911526 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4060fd-1a95-49b7-bb6f-af9e4f3e8c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:41:09.912148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4060fd-1a95-49b7-bb6f-af9e4f3e8c7f None admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:41:09.912463 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4060fd-1a95-49b7-bb6f-af9e4f3e8c7f None admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:41:09.912741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4060fd-1a95-49b7-bb6f-af9e4f3e8c7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:41:09.913447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4060fd-1a95-49b7-bb6f-af9e4f3e8c7f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:41:09.914819 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4060fd-1a95-49b7-bb6f-af9e4f3e8c7f None admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:41:09.934957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 659/1306] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:41:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:41:19.769057 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=30 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:41:19.931475 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:14.239289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad56b6b-a7bc-4912-9cef-444e8ae7d419 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:14.240196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad56b6b-a7bc-4912-9cef-444e8ae7d419 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:14.240650 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad56b6b-a7bc-4912-9cef-444e8ae7d419 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:14.247270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c574e-b384-48b1-9669-90be9a5d9c42 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:14.249206 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c574e-b384-48b1-9669-90be9a5d9c42 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:14.249771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c574e-b384-48b1-9669-90be9a5d9c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:14.255199 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 648/1307] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:42:14 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:14.392143 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d73c574e-b384-48b1-9669-90be9a5d9c42 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:14.396135 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d73c574e-b384-48b1-9669-90be9a5d9c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:14.535860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 660/1308] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:42:14.621891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d70a5-c640-4812-87fd-d1fee2d50290 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:14.622368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d70a5-c640-4812-87fd-d1fee2d50290 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:14.623571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d70a5-c640-4812-87fd-d1fee2d50290 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:14.625372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 649/1309] 10.222.0.69 () {54 vars in 965 bytes} [Wed Aug 3 18:42:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:14.631803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f17bb0-aff0-4f8c-9182-c890754d2330 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:14.632092 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f17bb0-aff0-4f8c-9182-c890754d2330 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:14.632399 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f17bb0-aff0-4f8c-9182-c890754d2330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:14.725170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-09f17bb0-aff0-4f8c-9182-c890754d2330 None None] MFA Rules not processed for user `0eb9ad672e5444098d71b7c9a27b72e8`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:14.726565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-09f17bb0-aff0-4f8c-9182-c890754d2330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:14.752850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:14.822613 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 661/1310] 10.222.0.69 () {58 vars in 1062 bytes} [Wed Aug 3 18:42:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:42:14.832690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d70a5-c640-4812-87fd-d1fee2d50290 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:42:14.834805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ab8d70a5-c640-4812-87fd-d1fee2d50290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:14.992832 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d70a5-c640-4812-87fd-d1fee2d50290 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:42:15.083270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc750ff7-d9e4-4cb2-909f-e38abbbccf0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:42:15.084043 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc750ff7-d9e4-4cb2-909f-e38abbbccf0f service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:15.084423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc750ff7-d9e4-4cb2-909f-e38abbbccf0f service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:15.084825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc750ff7-d9e4-4cb2-909f-e38abbbccf0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:15.086784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-cc750ff7-d9e4-4cb2-909f-e38abbbccf0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:15.153343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:15.226244 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc750ff7-d9e4-4cb2-909f-e38abbbccf0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:42:15.228446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc750ff7-d9e4-4cb2-909f-e38abbbccf0f service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:42:15.258494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 650/1311] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:42:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:42:17.709281 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cb954-006f-402b-9258-d3871e512738 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:17.709815 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cb954-006f-402b-9258-d3871e512738 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:17.709921 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cb954-006f-402b-9258-d3871e512738 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:17.711260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 662/1312] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:17.716898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7381376-0582-43e9-9f26-e1b39b5ee6fa None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:17.717414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7381376-0582-43e9-9f26-e1b39b5ee6fa None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:17.717414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7381376-0582-43e9-9f26-e1b39b5ee6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:17.798662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f7381376-0582-43e9-9f26-e1b39b5ee6fa None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:17.799869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f7381376-0582-43e9-9f26-e1b39b5ee6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:17.898556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 651/1313] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:42:17.937578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7cb954-006f-402b-9258-d3871e512738 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:42:17.939280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ff7cb954-006f-402b-9258-d3871e512738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:18.048299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7cb954-006f-402b-9258-d3871e512738 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:42:18.119583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:18.122908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d7168-1a76-4bd9-8018-5ed4aa245dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:42:18.123843 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d7168-1a76-4bd9-8018-5ed4aa245dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:18.124340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d7168-1a76-4bd9-8018-5ed4aa245dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:18.124822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d7168-1a76-4bd9-8018-5ed4aa245dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:18.126511 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a53d7168-1a76-4bd9-8018-5ed4aa245dc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:18.235766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d7168-1a76-4bd9-8018-5ed4aa245dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:42:18.237250 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d7168-1a76-4bd9-8018-5ed4aa245dc4 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:42:18.255820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 663/1314] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:42:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:42:20.563458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50babfd-c776-4785-806a-24818af95006 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:20.564001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50babfd-c776-4785-806a-24818af95006 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:20.564001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50babfd-c776-4785-806a-24818af95006 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:20.565417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 652/1315] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:20.570912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a294d1f-2e28-48be-8a62-f6bda6225414 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:20.571317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a294d1f-2e28-48be-8a62-f6bda6225414 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:20.571465 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a294d1f-2e28-48be-8a62-f6bda6225414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:20.637328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-8a294d1f-2e28-48be-8a62-f6bda6225414 None None] MFA Rules not processed for user `67faf276eb0a4b9baf8b596eb39d6bd2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:20.638655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8a294d1f-2e28-48be-8a62-f6bda6225414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:20.690405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:20.743023 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 664/1316] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:42:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:42:20.789319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50babfd-c776-4785-806a-24818af95006 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:42:20.790885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e50babfd-c776-4785-806a-24818af95006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:20.801385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:20.898176 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50babfd-c776-4785-806a-24818af95006 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:42:20.972527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291192fc-063c-443a-8710-d9a10b2831d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:42:20.973148 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291192fc-063c-443a-8710-d9a10b2831d0 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:20.973429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291192fc-063c-443a-8710-d9a10b2831d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:20.973702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291192fc-063c-443a-8710-d9a10b2831d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:20.975331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-291192fc-063c-443a-8710-d9a10b2831d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:21.037198 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:21.084921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291192fc-063c-443a-8710-d9a10b2831d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:42:21.086359 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291192fc-063c-443a-8710-d9a10b2831d0 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:42:21.105071 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 653/1317] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:42:20 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:42:23.668161 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d54b1-aa4d-4fbc-a287-7bfee99bda1b None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:23.668845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d54b1-aa4d-4fbc-a287-7bfee99bda1b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:23.668845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d54b1-aa4d-4fbc-a287-7bfee99bda1b None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:23.671273 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 665/1318] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:42:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:23.676397 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c532fce-aeff-4452-98f1-18160193785d None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:23.676840 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c532fce-aeff-4452-98f1-18160193785d None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:23.676937 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c532fce-aeff-4452-98f1-18160193785d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:23.741769 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-0c532fce-aeff-4452-98f1-18160193785d None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:23.742964 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-0c532fce-aeff-4452-98f1-18160193785d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:23.840615 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 654/1319] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:42:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:42:23.880201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d54b1-aa4d-4fbc-a287-7bfee99bda1b None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:42:23.881763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-9a2d54b1-aa4d-4fbc-a287-7bfee99bda1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:23.988181 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d54b1-aa4d-4fbc-a287-7bfee99bda1b None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:42:24.061799 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e28e2-fe55-4f03-839e-561759401315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:42:24.062417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e28e2-fe55-4f03-839e-561759401315 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:24.062700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e28e2-fe55-4f03-839e-561759401315 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:24.062975 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e28e2-fe55-4f03-839e-561759401315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:24.064656 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-299e28e2-fe55-4f03-839e-561759401315 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:24.072048 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:24.176350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e28e2-fe55-4f03-839e-561759401315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:42:24.177722 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e28e2-fe55-4f03-839e-561759401315 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:42:24.196103 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 666/1320] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:42:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:42:26.934197 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5d84e-5a9a-4d1c-962f-267b120c8cef None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:26.935296 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5d84e-5a9a-4d1c-962f-267b120c8cef None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:26.935444 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5d84e-5a9a-4d1c-962f-267b120c8cef None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:26.937114 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 655/1321] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:42:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:26.942453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e21337-d971-4fc1-a402-e85590603deb None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:26.942904 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e21337-d971-4fc1-a402-e85590603deb None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:26.942997 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e21337-d971-4fc1-a402-e85590603deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:27.010779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-18e21337-d971-4fc1-a402-e85590603deb None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:27.011978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-18e21337-d971-4fc1-a402-e85590603deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:27.077121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:27.120156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 667/1322] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:42:27.175087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d7561-a576-4e32-b5c6-014ee24b233c None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:27.175371 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d7561-a576-4e32-b5c6-014ee24b233c None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:27.175647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d7561-a576-4e32-b5c6-014ee24b233c None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:27.177089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 656/1323] 10.222.0.69 () {54 vars in 965 bytes} [Wed Aug 3 18:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:27.183669 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b3b66-6475-4447-b563-04a623777b5e None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:27.183957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b3b66-6475-4447-b563-04a623777b5e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:27.184321 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b3b66-6475-4447-b563-04a623777b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:27.269355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-2b6b3b66-6475-4447-b563-04a623777b5e None None] MFA Rules not processed for user `0eb9ad672e5444098d71b7c9a27b72e8`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:27.270553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2b6b3b66-6475-4447-b563-04a623777b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:27.348022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 668/1324] 10.222.0.69 () {58 vars in 1062 bytes} [Wed Aug 3 18:42:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:42:27.357286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d7561-a576-4e32-b5c6-014ee24b233c None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:42:27.358828 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-945d7561-a576-4e32-b5c6-014ee24b233c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:27.378520 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:27.468398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d7561-a576-4e32-b5c6-014ee24b233c None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:42:27.544679 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584d396-c90d-49cc-a822-08e137077359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:42:27.545359 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584d396-c90d-49cc-a822-08e137077359 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:27.545520 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584d396-c90d-49cc-a822-08e137077359 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:27.545795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584d396-c90d-49cc-a822-08e137077359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:27.547452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c584d396-c90d-49cc-a822-08e137077359 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:27.619573 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:27.658913 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584d396-c90d-49cc-a822-08e137077359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:42:27.660383 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584d396-c90d-49cc-a822-08e137077359 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:42:27.678838 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 657/1325] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:42:30.028439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39848f35-c638-4e08-9129-e7788412e7ca None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:30.029071 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39848f35-c638-4e08-9129-e7788412e7ca None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:30.029071 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39848f35-c638-4e08-9129-e7788412e7ca None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:30.030367 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 669/1326] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:30.035975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384b324-cf53-4383-bfb7-afb6d0116a31 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:30.036494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384b324-cf53-4383-bfb7-afb6d0116a31 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:30.036588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384b324-cf53-4383-bfb7-afb6d0116a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:30.103074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-8384b324-cf53-4383-bfb7-afb6d0116a31 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:30.104261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8384b324-cf53-4383-bfb7-afb6d0116a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:30.206650 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 658/1327] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:42:30.243446 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39848f35-c638-4e08-9129-e7788412e7ca None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:42:30.245052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-39848f35-c638-4e08-9129-e7788412e7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:30.308069 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:30.352567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39848f35-c638-4e08-9129-e7788412e7ca None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:42:30.429907 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13168b22-9709-4923-8f11-5db7cde9e795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:42:30.430536 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13168b22-9709-4923-8f11-5db7cde9e795 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:30.430816 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13168b22-9709-4923-8f11-5db7cde9e795 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:30.431088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13168b22-9709-4923-8f11-5db7cde9e795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:30.432751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-13168b22-9709-4923-8f11-5db7cde9e795 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:30.546619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:30.549091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13168b22-9709-4923-8f11-5db7cde9e795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:42:30.550875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13168b22-9709-4923-8f11-5db7cde9e795 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:42:30.570426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 670/1328] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:42:32.732763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e63b1a-630d-403b-b0ce-13f0a41b3707 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:32.732763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e63b1a-630d-403b-b0ce-13f0a41b3707 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:32.732763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e63b1a-630d-403b-b0ce-13f0a41b3707 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:32.734043 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 659/1329] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:32.739639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c13bc-c18a-4dd1-889f-481775a1b377 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:32.740176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c13bc-c18a-4dd1-889f-481775a1b377 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:32.740176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c13bc-c18a-4dd1-889f-481775a1b377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:32.806248 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-8c1c13bc-c18a-4dd1-889f-481775a1b377 None None] MFA Rules not processed for user `67faf276eb0a4b9baf8b596eb39d6bd2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:32.807453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8c1c13bc-c18a-4dd1-889f-481775a1b377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:32.912006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 671/1330] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:42:32 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:42:32.952202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e63b1a-630d-403b-b0ce-13f0a41b3707 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:42:32.953865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-37e63b1a-630d-403b-b0ce-13f0a41b3707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:32.986732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:33.068386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e63b1a-630d-403b-b0ce-13f0a41b3707 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:42:33.144112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109c575-3f79-4f8c-9f00-7dd911f7b740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:42:33.144335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109c575-3f79-4f8c-9f00-7dd911f7b740 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:33.144457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109c575-3f79-4f8c-9f00-7dd911f7b740 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:33.144714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109c575-3f79-4f8c-9f00-7dd911f7b740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:33.146328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e109c575-3f79-4f8c-9f00-7dd911f7b740 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:33.223474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:33.257329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109c575-3f79-4f8c-9f00-7dd911f7b740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:42:33.258813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109c575-3f79-4f8c-9f00-7dd911f7b740 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:42:33.278788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 660/1331] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:42:32 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:42:35.710482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3a5a8-9b79-4bde-9462-59acffad1421 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:35.711032 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3a5a8-9b79-4bde-9462-59acffad1421 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:35.711032 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3a5a8-9b79-4bde-9462-59acffad1421 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:35.712432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 672/1332] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:35.717841 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00c34e-d9b1-4ac3-b3d8-187ad3fc2f65 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:35.718254 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00c34e-d9b1-4ac3-b3d8-187ad3fc2f65 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:35.718396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00c34e-d9b1-4ac3-b3d8-187ad3fc2f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:35.790405 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ff00c34e-d9b1-4ac3-b3d8-187ad3fc2f65 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:35.791624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ff00c34e-d9b1-4ac3-b3d8-187ad3fc2f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:35.888547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 661/1333] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:42:35.924328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3a5a8-9b79-4bde-9462-59acffad1421 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:42:35.925860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2be3a5a8-9b79-4bde-9462-59acffad1421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:36.000583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:36.036226 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3a5a8-9b79-4bde-9462-59acffad1421 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:42:36.121411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab21c6a-6714-4a01-8a9b-1a1ab39c4083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:42:36.122554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab21c6a-6714-4a01-8a9b-1a1ab39c4083 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:36.123285 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab21c6a-6714-4a01-8a9b-1a1ab39c4083 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:36.123521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab21c6a-6714-4a01-8a9b-1a1ab39c4083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:36.125204 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-eab21c6a-6714-4a01-8a9b-1a1ab39c4083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:36.234596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab21c6a-6714-4a01-8a9b-1a1ab39c4083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:42:36.236034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab21c6a-6714-4a01-8a9b-1a1ab39c4083 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:42:36.246029 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:36.255722 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 673/1334] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:42:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:42:38.322074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7ff65-0166-427b-b4ba-9bdc3ef511f0 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:38.322639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7ff65-0166-427b-b4ba-9bdc3ef511f0 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:38.322639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7ff65-0166-427b-b4ba-9bdc3ef511f0 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:38.323978 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 662/1335] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:38.329219 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bd064-7ef8-4f48-b22f-b7a177a47542 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:38.329645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bd064-7ef8-4f48-b22f-b7a177a47542 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:38.329781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bd064-7ef8-4f48-b22f-b7a177a47542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:38.397100 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-723bd064-7ef8-4f48-b22f-b7a177a47542 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:38.398312 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-723bd064-7ef8-4f48-b22f-b7a177a47542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:38.497093 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 674/1336] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:42:38.541818 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7ff65-0166-427b-b4ba-9bdc3ef511f0 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:42:38.543356 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-94f7ff65-0166-427b-b4ba-9bdc3ef511f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:38.583541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:38.647390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7ff65-0166-427b-b4ba-9bdc3ef511f0 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:42:38.720265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d507a9-1906-4a4c-b5f3-18746e81d141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:42:38.720914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d507a9-1906-4a4c-b5f3-18746e81d141 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:38.721196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d507a9-1906-4a4c-b5f3-18746e81d141 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:38.721467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d507a9-1906-4a4c-b5f3-18746e81d141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:38.723097 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-44d507a9-1906-4a4c-b5f3-18746e81d141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:38.806234 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:38.832628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d507a9-1906-4a4c-b5f3-18746e81d141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:42:38.834121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d507a9-1906-4a4c-b5f3-18746e81d141 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:42:38.852215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 663/1337] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:42:41.338233 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ef362-90f1-46f5-b0b3-8e00d298e5eb None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:41.338806 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ef362-90f1-46f5-b0b3-8e00d298e5eb None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:41.338806 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ef362-90f1-46f5-b0b3-8e00d298e5eb None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:41.340559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 675/1338] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:41.345836 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfad940-11a9-4894-bea9-e75445c25d9d None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:41.346275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfad940-11a9-4894-bea9-e75445c25d9d None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:41.346386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfad940-11a9-4894-bea9-e75445c25d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:41.412995 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-5cfad940-11a9-4894-bea9-e75445c25d9d None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:41.414207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-5cfad940-11a9-4894-bea9-e75445c25d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:41.514076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 664/1339] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:42:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:42:41.550131 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ef362-90f1-46f5-b0b3-8e00d298e5eb None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:42:41.551683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b39ef362-90f1-46f5-b0b3-8e00d298e5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:41.629392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:41.657435 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ef362-90f1-46f5-b0b3-8e00d298e5eb None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:42:41.729546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707afb7d-9d15-4423-86c8-c3264e84d0a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:42:41.730159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707afb7d-9d15-4423-86c8-c3264e84d0a7 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:41.730440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707afb7d-9d15-4423-86c8-c3264e84d0a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:41.731340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707afb7d-9d15-4423-86c8-c3264e84d0a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:41.733044 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-707afb7d-9d15-4423-86c8-c3264e84d0a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:41.839668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707afb7d-9d15-4423-86c8-c3264e84d0a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:42:41.841725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707afb7d-9d15-4423-86c8-c3264e84d0a7 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:42:41.860252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 676/1340] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:42:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:42:45.891015 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9bfb8-887b-4a23-af62-1cc3bbf0976b None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:45.891547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9bfb8-887b-4a23-af62-1cc3bbf0976b None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:45.891547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9bfb8-887b-4a23-af62-1cc3bbf0976b None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:45.892953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 665/1341] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:45.898074 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b8e86-39db-4b63-900b-14a7d668c8e3 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:45.898528 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b8e86-39db-4b63-900b-14a7d668c8e3 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:45.898624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b8e86-39db-4b63-900b-14a7d668c8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:45.904913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:45.983885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-f20b8e86-39db-4b63-900b-14a7d668c8e3 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:45.985120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f20b8e86-39db-4b63-900b-14a7d668c8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:46.087836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 677/1342] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:42:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:42:46.127722 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9bfb8-887b-4a23-af62-1cc3bbf0976b None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:42:46.129343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-13a9bfb8-887b-4a23-af62-1cc3bbf0976b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:46.178848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:46.237278 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9bfb8-887b-4a23-af62-1cc3bbf0976b None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:42:46.315410 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d008d-44ce-4cbe-8dad-66d7602a1b1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:42:46.316035 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d008d-44ce-4cbe-8dad-66d7602a1b1d service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:46.316339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d008d-44ce-4cbe-8dad-66d7602a1b1d service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:46.316614 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d008d-44ce-4cbe-8dad-66d7602a1b1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:46.318219 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e24d008d-44ce-4cbe-8dad-66d7602a1b1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:46.422226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:46.442747 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d008d-44ce-4cbe-8dad-66d7602a1b1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:42:46.444318 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d008d-44ce-4cbe-8dad-66d7602a1b1d service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:42:46.465170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 666/1343] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:42:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:42:49.099776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6517992-2c1d-4d22-a137-779b733fcc70 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:49.100562 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6517992-2c1d-4d22-a137-779b733fcc70 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:49.100562 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6517992-2c1d-4d22-a137-779b733fcc70 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:49.101936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 678/1344] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:49.108194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb57abb-b706-470b-a7ca-830c276ebc05 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:49.108882 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb57abb-b706-470b-a7ca-830c276ebc05 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:49.108882 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb57abb-b706-470b-a7ca-830c276ebc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:49.183599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-3eb57abb-b706-470b-a7ca-830c276ebc05 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:49.185699 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-3eb57abb-b706-470b-a7ca-830c276ebc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:49.299338 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 667/1345] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:42:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:42:49.337865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6517992-2c1d-4d22-a137-779b733fcc70 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:42:49.339529 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a6517992-2c1d-4d22-a137-779b733fcc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:49.451468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:49.471799 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6517992-2c1d-4d22-a137-779b733fcc70 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:42:49.555301 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492d704-8bc0-4e79-aa69-d010a7d9ea3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:42:49.555897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492d704-8bc0-4e79-aa69-d010a7d9ea3d service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:49.556212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492d704-8bc0-4e79-aa69-d010a7d9ea3d service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:49.556896 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492d704-8bc0-4e79-aa69-d010a7d9ea3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:49.558992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-9492d704-8bc0-4e79-aa69-d010a7d9ea3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:49.673638 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492d704-8bc0-4e79-aa69-d010a7d9ea3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:42:49.675108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492d704-8bc0-4e79-aa69-d010a7d9ea3d service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:42:49.694374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 679/1346] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:42:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:42:53.848473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e66720-c4d8-43ec-87cc-c334dd55cd9f None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:53.849010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e66720-c4d8-43ec-87cc-c334dd55cd9f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:53.849010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e66720-c4d8-43ec-87cc-c334dd55cd9f None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:53.850431 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 668/1347] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:53.856015 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb497e-9b4c-4602-8b04-2f98c8771dad None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:53.856474 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb497e-9b4c-4602-8b04-2f98c8771dad None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:53.856697 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb497e-9b4c-4602-8b04-2f98c8771dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:53.873031 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:53.929456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-9ceb497e-9b4c-4602-8b04-2f98c8771dad None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:53.930803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-9ceb497e-9b4c-4602-8b04-2f98c8771dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:54.081451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 680/1348] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:42:54.118994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e66720-c4d8-43ec-87cc-c334dd55cd9f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:42:54.122479 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-60e66720-c4d8-43ec-87cc-c334dd55cd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:54.191294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:54.244861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e66720-c4d8-43ec-87cc-c334dd55cd9f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:42:54.331232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee3a61-db0e-4ad7-bee7-76ae20485d70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:42:54.332015 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee3a61-db0e-4ad7-bee7-76ae20485d70 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:54.332452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee3a61-db0e-4ad7-bee7-76ae20485d70 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:54.332849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee3a61-db0e-4ad7-bee7-76ae20485d70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:54.334761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-baee3a61-db0e-4ad7-bee7-76ae20485d70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:54.471825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:42:54.474353 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee3a61-db0e-4ad7-bee7-76ae20485d70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:42:54.476141 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee3a61-db0e-4ad7-bee7-76ae20485d70 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:42:54.498319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 669/1349] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:42:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:42:59.430215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44da3ac-fb03-4c91-b995-9252d6311cd1 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:59.430705 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44da3ac-fb03-4c91-b995-9252d6311cd1 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:59.430705 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44da3ac-fb03-4c91-b995-9252d6311cd1 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:59.432332 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 681/1350] 10.222.0.69 () {54 vars in 921 bytes} [Wed Aug 3 18:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:42:59.448185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1ab95-92a3-4e62-a8b3-c4453c88a18d None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:42:59.449198 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1ab95-92a3-4e62-a8b3-c4453c88a18d None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:42:59.449525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1ab95-92a3-4e62-a8b3-c4453c88a18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:42:59.605458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-6bc1ab95-92a3-4e62-a8b3-c4453c88a18d None None] MFA Rules not processed for user `fea93ceb4cbc49c2b4b280a4ddc257b7`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:42:59.607040 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-6bc1ab95-92a3-4e62-a8b3-c4453c88a18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:42:59.753093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 670/1351] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Aug 3 18:42:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:03.952873 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f171d8-fe38-43e7-b610-03f92334a9f5 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:03.953412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f171d8-fe38-43e7-b610-03f92334a9f5 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:03.953412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f171d8-fe38-43e7-b610-03f92334a9f5 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:03.955061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 682/1352] 10.222.0.69 () {54 vars in 921 bytes} [Wed Aug 3 18:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:03.960312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c859dbc-dce4-4797-a804-177c4be651ff None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:03.962491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c859dbc-dce4-4797-a804-177c4be651ff None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:03.962772 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c859dbc-dce4-4797-a804-177c4be651ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:04.022124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:04.049351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-6c859dbc-dce4-4797-a804-177c4be651ff None None] MFA Rules not processed for user `fea93ceb4cbc49c2b4b280a4ddc257b7`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:04.050552 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-6c859dbc-dce4-4797-a804-177c4be651ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:04.078189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=59 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:04.175045 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 671/1353] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Aug 3 18:43:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:43:05.617291 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24e01b-096c-4742-a89c-12ee0e919486 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:05.617845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24e01b-096c-4742-a89c-12ee0e919486 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:05.617845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24e01b-096c-4742-a89c-12ee0e919486 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:05.619227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 683/1354] 10.222.0.69 () {54 vars in 921 bytes} [Wed Aug 3 18:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:05.626234 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc71b41-e714-4193-9ede-6ee3864a39c9 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:05.626702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc71b41-e714-4193-9ede-6ee3864a39c9 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:05.626798 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc71b41-e714-4193-9ede-6ee3864a39c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:05.704638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-9bc71b41-e714-4193-9ede-6ee3864a39c9 None None] MFA Rules not processed for user `fea93ceb4cbc49c2b4b280a4ddc257b7`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:05.705837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9bc71b41-e714-4193-9ede-6ee3864a39c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:05.748039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:05.816402 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 672/1355] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Aug 3 18:43:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:09.237687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf354b2-7873-464d-982b-7d9938003303 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:09.238211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf354b2-7873-464d-982b-7d9938003303 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:09.238367 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf354b2-7873-464d-982b-7d9938003303 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:09.239798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 684/1356] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:09.244982 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328c177-8e7f-43ba-9ea9-6797628f4d93 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:09.245447 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328c177-8e7f-43ba-9ea9-6797628f4d93 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:09.245578 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328c177-8e7f-43ba-9ea9-6797628f4d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:09.331938 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-6328c177-8e7f-43ba-9ea9-6797628f4d93 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:09.334739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-6328c177-8e7f-43ba-9ea9-6797628f4d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:09.452981 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 673/1357] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:09.509373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c03b9e-1f08-4642-b2f9-0480be084164 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:09.510002 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c03b9e-1f08-4642-b2f9-0480be084164 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:09.510855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c03b9e-1f08-4642-b2f9-0480be084164 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:09.512943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 685/1358] 10.222.0.69 () {54 vars in 962 bytes} [Wed Aug 3 18:43:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:09.521747 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b52161-e542-4f67-8f37-8563c56ba42a None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:09.522657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b52161-e542-4f67-8f37-8563c56ba42a None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:09.523158 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b52161-e542-4f67-8f37-8563c56ba42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:09.544784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:09.614766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-56b52161-e542-4f67-8f37-8563c56ba42a None None] MFA Rules not processed for user `850a50cfe4394febbafb0908b08888b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:09.616732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-56b52161-e542-4f67-8f37-8563c56ba42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:09.752350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 674/1359] 10.222.0.69 () {58 vars in 1059 bytes} [Wed Aug 3 18:43:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:43:09.759857 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c03b9e-1f08-4642-b2f9-0480be084164 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:43:09.762012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-65c03b9e-1f08-4642-b2f9-0480be084164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:09.918915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c03b9e-1f08-4642-b2f9-0480be084164 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:43:10.132501 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:10.177970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b44f4-6bde-4fc5-acb0-5c252830b7fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:43:10.178599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b44f4-6bde-4fc5-acb0-5c252830b7fb service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:10.178882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b44f4-6bde-4fc5-acb0-5c252830b7fb service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:10.179158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b44f4-6bde-4fc5-acb0-5c252830b7fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:10.180906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-430b44f4-6bde-4fc5-acb0-5c252830b7fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:10.283399 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b44f4-6bde-4fc5-acb0-5c252830b7fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:43:10.285030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b44f4-6bde-4fc5-acb0-5c252830b7fb service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:43:10.305287 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 686/1360] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:43:12.581937 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c801b-eb4c-4107-bdb7-42813c5c89ad None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:12.582462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c801b-eb4c-4107-bdb7-42813c5c89ad None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:12.582574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c801b-eb4c-4107-bdb7-42813c5c89ad None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:12.584004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 675/1361] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:12.589257 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26aabf9-cb83-496c-9772-d52d092d7e9a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:12.590450 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26aabf9-cb83-496c-9772-d52d092d7e9a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:12.590859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26aabf9-cb83-496c-9772-d52d092d7e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:12.663567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-b26aabf9-cb83-496c-9772-d52d092d7e9a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:12.664955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b26aabf9-cb83-496c-9772-d52d092d7e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:12.709308 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:12.784346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 687/1362] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:12.833856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c801b-eb4c-4107-bdb7-42813c5c89ad None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:43:12.836591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-eb8c801b-eb4c-4107-bdb7-42813c5c89ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:12.924916 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:13.056672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c801b-eb4c-4107-bdb7-42813c5c89ad None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:43:13.189483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1a042-cc8d-4542-9eef-00cc1477a23b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:43:13.190416 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1a042-cc8d-4542-9eef-00cc1477a23b service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:13.190851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1a042-cc8d-4542-9eef-00cc1477a23b service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:13.191276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1a042-cc8d-4542-9eef-00cc1477a23b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:13.193223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-7ab1a042-cc8d-4542-9eef-00cc1477a23b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:13.220716 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:13.297646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1a042-cc8d-4542-9eef-00cc1477a23b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:43:13.299241 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1a042-cc8d-4542-9eef-00cc1477a23b service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:43:13.319158 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 676/1363] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:43:15.830917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce97f91-8335-4dc8-9b53-2b85a588d747 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:15.831483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce97f91-8335-4dc8-9b53-2b85a588d747 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:15.831483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce97f91-8335-4dc8-9b53-2b85a588d747 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:15.832841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 688/1364] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:15.838139 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa54f5-23f7-4cdc-8f0b-120a4726adc1 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:15.838545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa54f5-23f7-4cdc-8f0b-120a4726adc1 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:15.838730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa54f5-23f7-4cdc-8f0b-120a4726adc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:15.914857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-66aa54f5-23f7-4cdc-8f0b-120a4726adc1 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:15.914857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-66aa54f5-23f7-4cdc-8f0b-120a4726adc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:16.026194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:16.032326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 677/1365] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:43:16.080499 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440d0f5-7869-4c15-9875-2099597d60cb None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:16.081155 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440d0f5-7869-4c15-9875-2099597d60cb None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:16.081681 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440d0f5-7869-4c15-9875-2099597d60cb None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:16.083183 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 689/1366] 10.222.0.69 () {54 vars in 962 bytes} [Wed Aug 3 18:43:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:16.089372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce640f-198a-4bf8-8044-2765b588b0fe None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:16.090156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce640f-198a-4bf8-8044-2765b588b0fe None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:16.090156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce640f-198a-4bf8-8044-2765b588b0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:16.190528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-e2ce640f-198a-4bf8-8044-2765b588b0fe None None] MFA Rules not processed for user `850a50cfe4394febbafb0908b08888b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:16.191882 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e2ce640f-198a-4bf8-8044-2765b588b0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:16.329288 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 678/1367] 10.222.0.69 () {58 vars in 1059 bytes} [Wed Aug 3 18:43:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:16.338253 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440d0f5-7869-4c15-9875-2099597d60cb None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:43:16.341167 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-9440d0f5-7869-4c15-9875-2099597d60cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:16.502396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440d0f5-7869-4c15-9875-2099597d60cb None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:43:16.523850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:16.627893 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56d6b9-a5b5-4f01-9d71-0a63eca81d20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:43:16.628534 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56d6b9-a5b5-4f01-9d71-0a63eca81d20 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:16.632663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56d6b9-a5b5-4f01-9d71-0a63eca81d20 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:16.632663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56d6b9-a5b5-4f01-9d71-0a63eca81d20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:16.632663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-6b56d6b9-a5b5-4f01-9d71-0a63eca81d20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:16.736729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56d6b9-a5b5-4f01-9d71-0a63eca81d20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:43:16.738588 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56d6b9-a5b5-4f01-9d71-0a63eca81d20 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:43:16.759928 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 690/1368] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:43:17.658047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e3834-0be5-4e82-9333-5fca7b2c299c None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:17.659058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e3834-0be5-4e82-9333-5fca7b2c299c None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:17.659554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e3834-0be5-4e82-9333-5fca7b2c299c None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:17.661554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 679/1369] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:43:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:17.666301 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc44fff-db59-43eb-b964-66a16f3c4aa5 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:17.666708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc44fff-db59-43eb-b964-66a16f3c4aa5 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:17.666937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc44fff-db59-43eb-b964-66a16f3c4aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:17.699977 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:17.739219 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-cbc44fff-db59-43eb-b964-66a16f3c4aa5 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:17.740505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-cbc44fff-db59-43eb-b964-66a16f3c4aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:17.852246 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 691/1370] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:18.167903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd5a64-9919-4af4-9617-a8a8e7263bc2 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:18.168675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd5a64-9919-4af4-9617-a8a8e7263bc2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:18.169185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd5a64-9919-4af4-9617-a8a8e7263bc2 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:18.170967 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 680/1371] 10.222.0.69 () {54 vars in 964 bytes} [Wed Aug 3 18:43:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:18.176193 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd8dfb-fe7c-4337-ba85-8600ba865bab None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:18.176518 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd8dfb-fe7c-4337-ba85-8600ba865bab None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:18.176800 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd8dfb-fe7c-4337-ba85-8600ba865bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:18.265354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-90dd8dfb-fe7c-4337-ba85-8600ba865bab None None] MFA Rules not processed for user `c7d0fde945184c93b8fd91d15b66f534`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:18.266564 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-90dd8dfb-fe7c-4337-ba85-8600ba865bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:18.317559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:18.349470 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 692/1372] 10.222.0.69 () {58 vars in 1061 bytes} [Wed Aug 3 18:43:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:18.357633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd5a64-9919-4af4-9617-a8a8e7263bc2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:43:18.359681 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-91cd5a64-9919-4af4-9617-a8a8e7263bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:18.425802 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:18.489090 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd5a64-9919-4af4-9617-a8a8e7263bc2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:43:18.575726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2c5ed-d861-4d7d-8dbe-66d78ab0dcd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:43:18.576304 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2c5ed-d861-4d7d-8dbe-66d78ab0dcd1 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:18.576598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2c5ed-d861-4d7d-8dbe-66d78ab0dcd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:18.576876 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2c5ed-d861-4d7d-8dbe-66d78ab0dcd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:18.578633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-63c2c5ed-d861-4d7d-8dbe-66d78ab0dcd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:18.690364 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:18.707639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2c5ed-d861-4d7d-8dbe-66d78ab0dcd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:43:18.709212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2c5ed-d861-4d7d-8dbe-66d78ab0dcd1 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:43:18.730908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 681/1373] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:43:18.928433 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5c822-ac1e-4c28-8fce-41ea5d32a05a None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:18.928893 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5c822-ac1e-4c28-8fce-41ea5d32a05a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:18.929056 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5c822-ac1e-4c28-8fce-41ea5d32a05a None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:18.930342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 693/1374] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:18.937581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94958756-692b-41c7-84cd-962c56569d75 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:18.937820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94958756-692b-41c7-84cd-962c56569d75 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:18.938096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94958756-692b-41c7-84cd-962c56569d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:19.012958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-94958756-692b-41c7-84cd-962c56569d75 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:19.012958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-94958756-692b-41c7-84cd-962c56569d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:19.236233 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 682/1375] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:19.280356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5c822-ac1e-4c28-8fce-41ea5d32a05a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:43:19.281994 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e0b5c822-ac1e-4c28-8fce-41ea5d32a05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:19.433392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5c822-ac1e-4c28-8fce-41ea5d32a05a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:43:19.487095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:19.567566 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb363a-7f5f-4085-9684-aad8366b9a9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:43:19.568312 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb363a-7f5f-4085-9684-aad8366b9a9b service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:19.568675 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb363a-7f5f-4085-9684-aad8366b9a9b service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:19.568947 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb363a-7f5f-4085-9684-aad8366b9a9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:19.570566 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-59bb363a-7f5f-4085-9684-aad8366b9a9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:19.663204 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb363a-7f5f-4085-9684-aad8366b9a9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:43:19.665216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb363a-7f5f-4085-9684-aad8366b9a9b service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:43:19.686598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 694/1376] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:43:21.766776 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b6de8-e868-42e4-b0ed-ab5d4ddbe951 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:21.767320 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b6de8-e868-42e4-b0ed-ab5d4ddbe951 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:21.767320 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b6de8-e868-42e4-b0ed-ab5d4ddbe951 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:21.768702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 683/1377] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:21.773832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0071388-8b27-4407-a15d-c49c25fb4fb8 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:21.774236 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0071388-8b27-4407-a15d-c49c25fb4fb8 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:21.774391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0071388-8b27-4407-a15d-c49c25fb4fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:21.852201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-b0071388-8b27-4407-a15d-c49c25fb4fb8 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:21.854055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b0071388-8b27-4407-a15d-c49c25fb4fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:21.861785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:21.983915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 695/1378] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:22.030024 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b6de8-e868-42e4-b0ed-ab5d4ddbe951 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:43:22.031519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-161b6de8-e868-42e4-b0ed-ab5d4ddbe951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:22.095358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:22.184688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b6de8-e868-42e4-b0ed-ab5d4ddbe951 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:43:22.315155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df5b5e-351c-472e-b7af-98531e4354d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:43:22.315878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df5b5e-351c-472e-b7af-98531e4354d2 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:22.316230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df5b5e-351c-472e-b7af-98531e4354d2 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:22.316612 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df5b5e-351c-472e-b7af-98531e4354d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:22.318316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-61df5b5e-351c-472e-b7af-98531e4354d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:22.337296 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:22.440651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df5b5e-351c-472e-b7af-98531e4354d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:43:22.442426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df5b5e-351c-472e-b7af-98531e4354d2 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:43:22.468800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 684/1379] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:43:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:43:24.550949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01931832-01de-4e96-bf25-b3d7af47874a None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:24.551453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01931832-01de-4e96-bf25-b3d7af47874a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:24.551453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01931832-01de-4e96-bf25-b3d7af47874a None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:24.552988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 696/1380] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:24.558823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de454a-e15f-464d-a65e-9835991b35ba None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:24.558823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de454a-e15f-464d-a65e-9835991b35ba None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:24.559398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de454a-e15f-464d-a65e-9835991b35ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:24.630397 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f1de454a-e15f-464d-a65e-9835991b35ba None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:24.631585 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f1de454a-e15f-464d-a65e-9835991b35ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:24.729103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:24.737579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 685/1381] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:24.783222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01931832-01de-4e96-bf25-b3d7af47874a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:43:24.784878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-01931832-01de-4e96-bf25-b3d7af47874a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:24.940755 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:24.953477 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01931832-01de-4e96-bf25-b3d7af47874a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:43:25.076092 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561a88e-e5f2-4e1f-97bc-8b9634bdcdd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:43:25.076710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561a88e-e5f2-4e1f-97bc-8b9634bdcdd6 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:25.076998 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561a88e-e5f2-4e1f-97bc-8b9634bdcdd6 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:25.077274 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561a88e-e5f2-4e1f-97bc-8b9634bdcdd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:25.078900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f561a88e-e5f2-4e1f-97bc-8b9634bdcdd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:25.177040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561a88e-e5f2-4e1f-97bc-8b9634bdcdd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:43:25.178675 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561a88e-e5f2-4e1f-97bc-8b9634bdcdd6 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:43:25.190559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:25.204466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 697/1382] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:43:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:43:27.339313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5406a-2133-4220-a583-6e5e094667c5 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:27.340093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5406a-2133-4220-a583-6e5e094667c5 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:27.340093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5406a-2133-4220-a583-6e5e094667c5 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:27.341333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 686/1383] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:27.346743 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484e982-d175-4082-9ad1-19294a335265 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:27.347190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484e982-d175-4082-9ad1-19294a335265 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:27.347390 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484e982-d175-4082-9ad1-19294a335265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:27.419268 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-a484e982-d175-4082-9ad1-19294a335265 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:27.420499 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a484e982-d175-4082-9ad1-19294a335265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:27.521281 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 698/1384] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:43:27.567051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5406a-2133-4220-a583-6e5e094667c5 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:43:27.569631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-08d5406a-2133-4220-a583-6e5e094667c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:27.741104 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5406a-2133-4220-a583-6e5e094667c5 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:43:27.813240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:27.885704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483dd1e-8071-49f0-9ab0-fb0589ef2b33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:43:27.886847 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483dd1e-8071-49f0-9ab0-fb0589ef2b33 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:27.887313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483dd1e-8071-49f0-9ab0-fb0589ef2b33 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:27.887726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483dd1e-8071-49f0-9ab0-fb0589ef2b33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:27.890390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a483dd1e-8071-49f0-9ab0-fb0589ef2b33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:28.008329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483dd1e-8071-49f0-9ab0-fb0589ef2b33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:43:28.010884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483dd1e-8071-49f0-9ab0-fb0589ef2b33 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:43:28.041813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 687/1385] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:43:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:43:30.426684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2badd76-7978-483e-ab1e-6d1af68029af None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:30.427220 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2badd76-7978-483e-ab1e-6d1af68029af None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:30.427519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2badd76-7978-483e-ab1e-6d1af68029af None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:30.429834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 699/1386] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:43:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:30.435227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0920a0c-659a-4e39-ab53-85582565674b None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:30.435666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0920a0c-659a-4e39-ab53-85582565674b None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:30.435857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0920a0c-659a-4e39-ab53-85582565674b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:30.503756 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-e0920a0c-659a-4e39-ab53-85582565674b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:30.505000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e0920a0c-659a-4e39-ab53-85582565674b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:30.546036 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:30.614725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 688/1387] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:30.662164 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2badd76-7978-483e-ab1e-6d1af68029af None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:43:30.663784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d2badd76-7978-483e-ab1e-6d1af68029af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:30.748346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:30.820497 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2badd76-7978-483e-ab1e-6d1af68029af None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:43:30.952506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e50d13-d903-4847-9831-a5bf55c32809 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:43:30.953556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e50d13-d903-4847-9831-a5bf55c32809 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:30.954078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e50d13-d903-4847-9831-a5bf55c32809 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:30.954421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e50d13-d903-4847-9831-a5bf55c32809 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:30.956541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c4e50d13-d903-4847-9831-a5bf55c32809 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:31.008827 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:31.078309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e50d13-d903-4847-9831-a5bf55c32809 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:43:31.080478 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e50d13-d903-4847-9831-a5bf55c32809 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:43:31.103787 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 700/1388] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:43:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:43:33.222539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b25d8c-71e8-41f8-9b1d-edfa48c87b80 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:33.223136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b25d8c-71e8-41f8-9b1d-edfa48c87b80 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:33.223136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b25d8c-71e8-41f8-9b1d-edfa48c87b80 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:33.224675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 689/1389] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:33.229840 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb3a0b-6952-4da0-b7b2-4cabba58adee None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:33.231306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb3a0b-6952-4da0-b7b2-4cabba58adee None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:33.231595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb3a0b-6952-4da0-b7b2-4cabba58adee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:33.300018 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-70eb3a0b-6952-4da0-b7b2-4cabba58adee None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:33.301230 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-70eb3a0b-6952-4da0-b7b2-4cabba58adee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:33.405525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 701/1390] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:33.450324 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b25d8c-71e8-41f8-9b1d-edfa48c87b80 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:43:33.451905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-09b25d8c-71e8-41f8-9b1d-edfa48c87b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:33.605201 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b25d8c-71e8-41f8-9b1d-edfa48c87b80 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:43:33.622734 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:33.720845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250b2e8-dfb3-433e-af7c-d7ce632fe1a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:43:33.721506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250b2e8-dfb3-433e-af7c-d7ce632fe1a3 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:33.721792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250b2e8-dfb3-433e-af7c-d7ce632fe1a3 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:33.722073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250b2e8-dfb3-433e-af7c-d7ce632fe1a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:33.723744 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9250b2e8-dfb3-433e-af7c-d7ce632fe1a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:33.817071 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250b2e8-dfb3-433e-af7c-d7ce632fe1a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:43:33.818664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250b2e8-dfb3-433e-af7c-d7ce632fe1a3 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:43:33.837722 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 690/1391] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:43:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:43:34.105583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f323b4-f729-4784-aa36-7574c6d4a2fe None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:34.105869 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f323b4-f729-4784-aa36-7574c6d4a2fe None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:34.106154 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f323b4-f729-4784-aa36-7574c6d4a2fe None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:34.107532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 702/1392] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:34.112767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cbc76-d0d8-4df8-bf6a-cb99d76f97fe None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:34.113098 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cbc76-d0d8-4df8-bf6a-cb99d76f97fe None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:34.113322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cbc76-d0d8-4df8-bf6a-cb99d76f97fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:34.139810 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:34.182565 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-d27cbc76-d0d8-4df8-bf6a-cb99d76f97fe None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:34.183969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d27cbc76-d0d8-4df8-bf6a-cb99d76f97fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:34.288574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 691/1393] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:34.346803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39591f-08b4-4951-8b5b-879bcbf62343 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:34.347314 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39591f-08b4-4951-8b5b-879bcbf62343 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:34.347425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39591f-08b4-4951-8b5b-879bcbf62343 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:34.348946 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 703/1394] 10.222.0.69 () {54 vars in 966 bytes} [Wed Aug 3 18:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:34.353979 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cef87-3d0d-4d19-8046-c48b131e2244 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:34.354263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cef87-3d0d-4d19-8046-c48b131e2244 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:34.354535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cef87-3d0d-4d19-8046-c48b131e2244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:34.434194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-044cef87-3d0d-4d19-8046-c48b131e2244 None None] MFA Rules not processed for user `4799b0c9e5aa47d389121d2183666d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:34.435459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-044cef87-3d0d-4d19-8046-c48b131e2244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:34.476995 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:34.516116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 692/1395] 10.222.0.69 () {58 vars in 1063 bytes} [Wed Aug 3 18:43:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:34.523851 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39591f-08b4-4951-8b5b-879bcbf62343 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:43:34.525401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ca39591f-08b4-4951-8b5b-879bcbf62343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:34.539359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:34.643729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39591f-08b4-4951-8b5b-879bcbf62343 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:43:34.717646 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123a2c9-c3b8-47c3-bcc2-7c14478127f4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4799b0c9e5aa47d389121d2183666d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:43:34.718276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123a2c9-c3b8-47c3-bcc2-7c14478127f4 service glance] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:34.718555 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123a2c9-c3b8-47c3-bcc2-7c14478127f4 service glance] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:34.718830 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123a2c9-c3b8-47c3-bcc2-7c14478127f4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:34.720485 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b123a2c9-c3b8-47c3-bcc2-7c14478127f4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:34.788144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:34.830587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123a2c9-c3b8-47c3-bcc2-7c14478127f4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:43:34.832037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123a2c9-c3b8-47c3-bcc2-7c14478127f4 service glance] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:43:34.853603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 704/1396] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:43:35.603335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1499c2-4ece-4c08-be0f-7754731f4a5e None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:35.603804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1499c2-4ece-4c08-be0f-7754731f4a5e None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:35.603898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1499c2-4ece-4c08-be0f-7754731f4a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:35.676736 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-4b1499c2-4ece-4c08-be0f-7754731f4a5e None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:35.677956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4b1499c2-4ece-4c08-be0f-7754731f4a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:35.767513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 693/1397] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Aug 3 18:43:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:35.812308 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c93287-a87c-44a1-8a68-090bb8e2215f None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:35.812817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c93287-a87c-44a1-8a68-090bb8e2215f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:35.812926 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c93287-a87c-44a1-8a68-090bb8e2215f None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:35.814307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 705/1398] 10.222.0.69 () {54 vars in 943 bytes} [Wed Aug 3 18:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:35.820651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60914f48-9dbd-4337-a4fd-0c3a61c77a55 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:35.821007 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60914f48-9dbd-4337-a4fd-0c3a61c77a55 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:35.821282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60914f48-9dbd-4337-a4fd-0c3a61c77a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:35.893690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:35.904190 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-60914f48-9dbd-4337-a4fd-0c3a61c77a55 None None] MFA Rules not processed for user `96ed059287274ca392c6abb893551094`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:35.905627 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-60914f48-9dbd-4337-a4fd-0c3a61c77a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:35.918832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4c206-d403-4af8-b478-a1596912f235 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:35.919114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4c206-d403-4af8-b478-a1596912f235 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:35.919419 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4c206-d403-4af8-b478-a1596912f235 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:35.920816 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 706/1399] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:43:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:35.925914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e75de1-afd5-43ce-b9a0-cd080e633448 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:35.926206 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e75de1-afd5-43ce-b9a0-cd080e633448 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:35.926480 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e75de1-afd5-43ce-b9a0-cd080e633448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:35.996710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-92e75de1-afd5-43ce-b9a0-cd080e633448 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:35.998272 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-92e75de1-afd5-43ce-b9a0-cd080e633448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:36.042453 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 694/1400] 10.222.0.69 () {58 vars in 1040 bytes} [Wed Aug 3 18:43:35 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:36.053137 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60914f48-9dbd-4337-a4fd-0c3a61c77a55 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:43:36.063117 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60914f48-9dbd-4337-a4fd-0c3a61c77a55 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:43:36.100908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 707/1401] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:43:36.146498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e75de1-afd5-43ce-b9a0-cd080e633448 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:43:36.147728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-92e75de1-afd5-43ce-b9a0-cd080e633448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:36.172599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929574c-974f-4b40-b507-7fd678115920 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:43:36.173260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929574c-974f-4b40-b507-7fd678115920 service swift] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:36.173544 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929574c-974f-4b40-b507-7fd678115920 service swift] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:36.173817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929574c-974f-4b40-b507-7fd678115920 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:36.175140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:36.175869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-7929574c-974f-4b40-b507-7fd678115920 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:36.178538 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:36.263823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929574c-974f-4b40-b507-7fd678115920 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:43:36.265641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929574c-974f-4b40-b507-7fd678115920 service swift] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:43:36.267179 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e75de1-afd5-43ce-b9a0-cd080e633448 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:43:36.274470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 695/1402] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 18:43:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 18:43:36.417034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562db8d-69e2-4d3a-973f-6619ca168ea7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:43:36.418237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562db8d-69e2-4d3a-973f-6619ca168ea7 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:36.418792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562db8d-69e2-4d3a-973f-6619ca168ea7 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:36.419131 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562db8d-69e2-4d3a-973f-6619ca168ea7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:36.420907 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1562db8d-69e2-4d3a-973f-6619ca168ea7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:36.441487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:36.525268 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562db8d-69e2-4d3a-973f-6619ca168ea7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:43:36.527436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562db8d-69e2-4d3a-973f-6619ca168ea7 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:43:36.556509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 708/1403] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:43:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:43:38.651173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3589cb-6552-4fb4-95f3-e0a69c4c8ff5 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:38.652835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3589cb-6552-4fb4-95f3-e0a69c4c8ff5 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:38.653131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3589cb-6552-4fb4-95f3-e0a69c4c8ff5 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:38.654556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 696/1404] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:43:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:38.659968 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eefb77-558a-4fa9-bc13-2f483396a65f None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:38.660445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eefb77-558a-4fa9-bc13-2f483396a65f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:38.660639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eefb77-558a-4fa9-bc13-2f483396a65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:38.735577 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-51eefb77-558a-4fa9-bc13-2f483396a65f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:38.737153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-51eefb77-558a-4fa9-bc13-2f483396a65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:38.841652 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:38.847706 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 709/1405] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:38.894165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3589cb-6552-4fb4-95f3-e0a69c4c8ff5 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:43:38.895678 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-7c3589cb-6552-4fb4-95f3-e0a69c4c8ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:39.068127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:39.071458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3589cb-6552-4fb4-95f3-e0a69c4c8ff5 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:43:39.203975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5a0a9-791b-49dd-b797-2d4980fb5f29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:43:39.204645 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5a0a9-791b-49dd-b797-2d4980fb5f29 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:39.204921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5a0a9-791b-49dd-b797-2d4980fb5f29 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:39.205193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5a0a9-791b-49dd-b797-2d4980fb5f29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:39.206858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-91e5a0a9-791b-49dd-b797-2d4980fb5f29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:39.327435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5a0a9-791b-49dd-b797-2d4980fb5f29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:43:39.330901 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5a0a9-791b-49dd-b797-2d4980fb5f29 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:43:39.352189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 697/1406] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:43:41.488934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7023c52-03ab-4c1e-8b90-818f07522969 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:41.489510 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7023c52-03ab-4c1e-8b90-818f07522969 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:41.489510 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7023c52-03ab-4c1e-8b90-818f07522969 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:41.492631 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 710/1407] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:43:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:41.496489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4aa1a1-90eb-4df1-b760-a867dbea2421 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:41.497679 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4aa1a1-90eb-4df1-b760-a867dbea2421 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:41.497965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4aa1a1-90eb-4df1-b760-a867dbea2421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:41.502787 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:41.578866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-6b4aa1a1-90eb-4df1-b760-a867dbea2421 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:41.578866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-6b4aa1a1-90eb-4df1-b760-a867dbea2421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:41.692439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 698/1408] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:41.739984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7023c52-03ab-4c1e-8b90-818f07522969 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:43:41.742629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f7023c52-03ab-4c1e-8b90-818f07522969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:41.916062 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7023c52-03ab-4c1e-8b90-818f07522969 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:43:42.001588 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:42.056929 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5638f19-ca4e-4afe-b027-29a896f7ad52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:43:42.057614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5638f19-ca4e-4afe-b027-29a896f7ad52 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:42.057991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5638f19-ca4e-4afe-b027-29a896f7ad52 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:42.059100 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5638f19-ca4e-4afe-b027-29a896f7ad52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:42.061327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d5638f19-ca4e-4afe-b027-29a896f7ad52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:42.167014 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5638f19-ca4e-4afe-b027-29a896f7ad52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:43:42.168950 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5638f19-ca4e-4afe-b027-29a896f7ad52 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:43:42.192535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 711/1409] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:43:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:43:51.687900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=64 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:52.187984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=64 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:58.904398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fecdc-b5ec-4983-872c-36f73d410506 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:58.904983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fecdc-b5ec-4983-872c-36f73d410506 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:58.904983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fecdc-b5ec-4983-872c-36f73d410506 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:58.906305 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 699/1410] 10.222.0.69 () {54 vars in 918 bytes} [Wed Aug 3 18:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:58.911335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234347a3-0db7-4138-b8f9-6fdc89e122a7 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:58.911735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234347a3-0db7-4138-b8f9-6fdc89e122a7 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:58.911886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234347a3-0db7-4138-b8f9-6fdc89e122a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:58.988892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-234347a3-0db7-4138-b8f9-6fdc89e122a7 None None] MFA Rules not processed for user `fea93ceb4cbc49c2b4b280a4ddc257b7`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:58.990108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-234347a3-0db7-4138-b8f9-6fdc89e122a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:59.097738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 712/1411] 10.222.0.69 () {58 vars in 1015 bytes} [Wed Aug 3 18:43:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:43:59.113124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf121b-ba97-4612-aec2-c585a1c55bfc None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:59.113413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf121b-ba97-4612-aec2-c585a1c55bfc None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:59.113693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf121b-ba97-4612-aec2-c585a1c55bfc None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:59.115042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 700/1412] 10.222.0.69 () {54 vars in 918 bytes} [Wed Aug 3 18:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:43:59.119548 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00af1f-1061-460d-b2ed-945a59219828 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:43:59.119837 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00af1f-1061-460d-b2ed-945a59219828 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:43:59.120124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00af1f-1061-460d-b2ed-945a59219828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:43:59.196893 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:43:59.199885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-1a00af1f-1061-460d-b2ed-945a59219828 None None] MFA Rules not processed for user `850a50cfe4394febbafb0908b08888b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:43:59.201714 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1a00af1f-1061-460d-b2ed-945a59219828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:43:59.325400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 713/1413] 10.222.0.69 () {58 vars in 1015 bytes} [Wed Aug 3 18:43:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:01.823002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a8743-127d-4028-a3cd-995a498e01fa None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:01.823552 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a8743-127d-4028-a3cd-995a498e01fa None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:01.823552 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a8743-127d-4028-a3cd-995a498e01fa None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:01.825049 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 18:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:01.830280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a035b9e-d688-4193-ad8d-1ca34e6b787d None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:01.830696 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a035b9e-d688-4193-ad8d-1ca34e6b787d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:01.830824 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a035b9e-d688-4193-ad8d-1ca34e6b787d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:01.896894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-5a035b9e-d688-4193-ad8d-1ca34e6b787d None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:01.898101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5a035b9e-d688-4193-ad8d-1ca34e6b787d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:01.954386 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:01.999433 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 18:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:02.042422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a8743-127d-4028-a3cd-995a498e01fa None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:02.045082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f83a8743-127d-4028-a3cd-995a498e01fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:02.193439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a8743-127d-4028-a3cd-995a498e01fa None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:02.267023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:02.307552 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e36a5d-dc9b-4192-a43d-c81df2e7b8ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:02.308179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e36a5d-dc9b-4192-a43d-c81df2e7b8ec service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:02.309421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e36a5d-dc9b-4192-a43d-c81df2e7b8ec service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:02.309421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e36a5d-dc9b-4192-a43d-c81df2e7b8ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:02.311866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-02e36a5d-dc9b-4192-a43d-c81df2e7b8ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:02.403640 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e36a5d-dc9b-4192-a43d-c81df2e7b8ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:02.405256 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e36a5d-dc9b-4192-a43d-c81df2e7b8ec service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:02.423577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 702/1416] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:03.330563 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82529f-5c63-4656-b0e7-ca515a8a5214 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:03.331128 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82529f-5c63-4656-b0e7-ca515a8a5214 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:03.331212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82529f-5c63-4656-b0e7-ca515a8a5214 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:03.332627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 18:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:03.338460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6ce49-f18b-4a4a-b1b8-6888621503a9 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:03.338922 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6ce49-f18b-4a4a-b1b8-6888621503a9 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:03.339156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6ce49-f18b-4a4a-b1b8-6888621503a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:03.415455 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-67b6ce49-f18b-4a4a-b1b8-6888621503a9 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:03.416696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-67b6ce49-f18b-4a4a-b1b8-6888621503a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:03.444697 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:03.525936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 18:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:03.679620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82529f-5c63-4656-b0e7-ca515a8a5214 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:03.681086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5b82529f-5c63-4656-b0e7-ca515a8a5214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:03.835130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82529f-5c63-4656-b0e7-ca515a8a5214 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:03.877063 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:03.954362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5316d-6ced-41d5-a977-dc4c178f2869 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:03.954982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5316d-6ced-41d5-a977-dc4c178f2869 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:03.955259 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5316d-6ced-41d5-a977-dc4c178f2869 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:03.955529 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5316d-6ced-41d5-a977-dc4c178f2869 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:03.957182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-32c5316d-6ced-41d5-a977-dc4c178f2869 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:04.047855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5316d-6ced-41d5-a977-dc4c178f2869 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:04.049483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5316d-6ced-41d5-a977-dc4c178f2869 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:04.067883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 716/1419] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:04.971848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605aa9b-54fc-4a29-86c7-cd8731acf9d7 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:04.972391 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605aa9b-54fc-4a29-86c7-cd8731acf9d7 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:04.972492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605aa9b-54fc-4a29-86c7-cd8731acf9d7 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:04.973802 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 18:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:04.978794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ff9ff-d180-4035-b806-bc854e6a2f4c None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:04.979229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ff9ff-d180-4035-b806-bc854e6a2f4c None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:04.979328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ff9ff-d180-4035-b806-bc854e6a2f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:05.035098 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:05.057170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-5b3ff9ff-d180-4035-b806-bc854e6a2f4c None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:05.058610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5b3ff9ff-d180-4035-b806-bc854e6a2f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:05.156705 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 18:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:44:05.314568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605aa9b-54fc-4a29-86c7-cd8731acf9d7 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:05.316097 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c605aa9b-54fc-4a29-86c7-cd8731acf9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:05.463790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605aa9b-54fc-4a29-86c7-cd8731acf9d7 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:05.476537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:05.581319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405ef62-49f9-4889-833f-ed60590a6ae9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:05.581905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405ef62-49f9-4889-833f-ed60590a6ae9 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:05.582247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405ef62-49f9-4889-833f-ed60590a6ae9 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:05.582525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405ef62-49f9-4889-833f-ed60590a6ae9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:05.584156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f405ef62-49f9-4889-833f-ed60590a6ae9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:05.674096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405ef62-49f9-4889-833f-ed60590a6ae9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:05.675687 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405ef62-49f9-4889-833f-ed60590a6ae9 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:05.694587 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 705/1422] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:06.589445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e1c26-f553-4f5b-9e70-426a4a0ce37d None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:06.592104 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e1c26-f553-4f5b-9e70-426a4a0ce37d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:06.592104 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e1c26-f553-4f5b-9e70-426a4a0ce37d None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:06.592314 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 18:44:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:06.597171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7e7a1-d70a-44c9-97f8-08992dfdd4f4 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:06.597635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7e7a1-d70a-44c9-97f8-08992dfdd4f4 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:06.597727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7e7a1-d70a-44c9-97f8-08992dfdd4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:06.617395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:06.676727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-38c7e7a1-d70a-44c9-97f8-08992dfdd4f4 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:06.677940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-38c7e7a1-d70a-44c9-97f8-08992dfdd4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:06.776721 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 18:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:06.928200 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e1c26-f553-4f5b-9e70-426a4a0ce37d None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:06.929708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0d8e1c26-f553-4f5b-9e70-426a4a0ce37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:07.042169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:07.078559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e1c26-f553-4f5b-9e70-426a4a0ce37d None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:07.190612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c43fe5-e537-4489-9b18-c5f12f8a9d32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:07.191181 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c43fe5-e537-4489-9b18-c5f12f8a9d32 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:07.191463 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c43fe5-e537-4489-9b18-c5f12f8a9d32 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:07.191738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c43fe5-e537-4489-9b18-c5f12f8a9d32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:07.193403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-16c43fe5-e537-4489-9b18-c5f12f8a9d32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:07.273209 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:07.289941 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c43fe5-e537-4489-9b18-c5f12f8a9d32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:07.291566 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c43fe5-e537-4489-9b18-c5f12f8a9d32 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:07.311294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 719/1425] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:08.233142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153397e-7955-4b42-899f-03695cce22b6 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:08.233790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153397e-7955-4b42-899f-03695cce22b6 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:08.233790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153397e-7955-4b42-899f-03695cce22b6 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:08.235066 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 18:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:08.240094 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f8e1-e35a-4cb2-a875-fb95195d3768 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:08.240547 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f8e1-e35a-4cb2-a875-fb95195d3768 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:08.240676 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f8e1-e35a-4cb2-a875-fb95195d3768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:08.316981 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-6e93f8e1-e35a-4cb2-a875-fb95195d3768 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:08.318198 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-6e93f8e1-e35a-4cb2-a875-fb95195d3768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:08.415931 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 18:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:08.567492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153397e-7955-4b42-899f-03695cce22b6 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:08.568959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1153397e-7955-4b42-899f-03695cce22b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:08.655297 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:08.718757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153397e-7955-4b42-899f-03695cce22b6 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:08.832624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b77e8-47f6-4717-978d-d2d39cff6d38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:08.833194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b77e8-47f6-4717-978d-d2d39cff6d38 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:08.833477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b77e8-47f6-4717-978d-d2d39cff6d38 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:08.833774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b77e8-47f6-4717-978d-d2d39cff6d38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:08.835435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-998b77e8-47f6-4717-978d-d2d39cff6d38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:08.883807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:08.927863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b77e8-47f6-4717-978d-d2d39cff6d38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:08.929541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b77e8-47f6-4717-978d-d2d39cff6d38 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:08.948031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 708/1428] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:09.801053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a874a-2166-45a2-848f-dbfac0839050 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:09.801628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a874a-2166-45a2-848f-dbfac0839050 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:09.801738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a874a-2166-45a2-848f-dbfac0839050 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:09.803075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 18:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:09.808044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d20ea-dcbb-4a50-91e1-92d6ac21a506 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:09.808571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d20ea-dcbb-4a50-91e1-92d6ac21a506 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:09.809168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d20ea-dcbb-4a50-91e1-92d6ac21a506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:09.885145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-587d20ea-dcbb-4a50-91e1-92d6ac21a506 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:09.886389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-587d20ea-dcbb-4a50-91e1-92d6ac21a506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:09.983083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 18:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:10.134371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a874a-2166-45a2-848f-dbfac0839050 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:10.135808 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-092a874a-2166-45a2-848f-dbfac0839050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:10.185933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:10.285769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a874a-2166-45a2-848f-dbfac0839050 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:10.402069 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de56966-4072-430c-8665-4852f1231fac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:10.402601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de56966-4072-430c-8665-4852f1231fac service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:10.402887 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de56966-4072-430c-8665-4852f1231fac service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:10.403212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de56966-4072-430c-8665-4852f1231fac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:10.405437 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8de56966-4072-430c-8665-4852f1231fac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:10.412541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:10.497952 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de56966-4072-430c-8665-4852f1231fac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:10.499549 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de56966-4072-430c-8665-4852f1231fac service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:10.518404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 722/1431] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:11.330459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f10b7-c9b5-49d9-a137-470427503fa9 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:11.331027 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f10b7-c9b5-49d9-a137-470427503fa9 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:11.331129 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f10b7-c9b5-49d9-a137-470427503fa9 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:11.332658 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 18:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:11.337675 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c728c69-e873-4b1e-a1b6-ca8b3f0a088d None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:11.338100 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c728c69-e873-4b1e-a1b6-ca8b3f0a088d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:11.338227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c728c69-e873-4b1e-a1b6-ca8b3f0a088d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:11.417967 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-0c728c69-e873-4b1e-a1b6-ca8b3f0a088d None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:11.419194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0c728c69-e873-4b1e-a1b6-ca8b3f0a088d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:11.487229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:11.516973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 18:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:11.669477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f10b7-c9b5-49d9-a137-470427503fa9 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:11.671445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b18f10b7-c9b5-49d9-a137-470427503fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:11.691228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:11.832411 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f10b7-c9b5-49d9-a137-470427503fa9 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:11.921885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:11.948576 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a58d1-efb7-4115-8490-acce08ba5501 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:11.949852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a58d1-efb7-4115-8490-acce08ba5501 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:11.950193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a58d1-efb7-4115-8490-acce08ba5501 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:11.950470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a58d1-efb7-4115-8490-acce08ba5501 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:11.952132 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-995a58d1-efb7-4115-8490-acce08ba5501 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:12.048333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a58d1-efb7-4115-8490-acce08ba5501 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:12.049873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a58d1-efb7-4115-8490-acce08ba5501 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:12.070340 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 711/1434] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:12.878425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fd2c7-79cc-40f1-bc30-dca1aeac3ec8 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:12.878934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fd2c7-79cc-40f1-bc30-dca1aeac3ec8 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:12.879054 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fd2c7-79cc-40f1-bc30-dca1aeac3ec8 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:12.880409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 18:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:12.885450 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f694c3-45ee-4d27-a8d0-5443df051b23 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:12.885880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f694c3-45ee-4d27-a8d0-5443df051b23 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:12.886036 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f694c3-45ee-4d27-a8d0-5443df051b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:12.963361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-23f694c3-45ee-4d27-a8d0-5443df051b23 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:12.964607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-23f694c3-45ee-4d27-a8d0-5443df051b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:13.006408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:13.063636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 18:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:13.219820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fd2c7-79cc-40f1-bc30-dca1aeac3ec8 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:13.221374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f17fd2c7-79cc-40f1-bc30-dca1aeac3ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:13.375526 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fd2c7-79cc-40f1-bc30-dca1aeac3ec8 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:13.431571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:13.493423 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e98593-1fba-48df-8d54-94adbd6f2938 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:13.493960 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e98593-1fba-48df-8d54-94adbd6f2938 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:13.494238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e98593-1fba-48df-8d54-94adbd6f2938 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:13.494507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e98593-1fba-48df-8d54-94adbd6f2938 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:13.497163 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a0e98593-1fba-48df-8d54-94adbd6f2938 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:13.588799 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e98593-1fba-48df-8d54-94adbd6f2938 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:13.590804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e98593-1fba-48df-8d54-94adbd6f2938 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:13.609021 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 725/1437] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:14.424793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd076d-37df-4fa9-af39-ce15ea631de2 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:14.425333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd076d-37df-4fa9-af39-ce15ea631de2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:14.425333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd076d-37df-4fa9-af39-ce15ea631de2 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:14.426727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 18:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:14.431784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7d9a6-dd60-4a34-98e8-195e71fca933 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:14.432213 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7d9a6-dd60-4a34-98e8-195e71fca933 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:14.432364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7d9a6-dd60-4a34-98e8-195e71fca933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:14.517896 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-50a7d9a6-dd60-4a34-98e8-195e71fca933 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:14.519102 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-50a7d9a6-dd60-4a34-98e8-195e71fca933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:14.521108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:14.631753 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 18:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:14.785521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd076d-37df-4fa9-af39-ce15ea631de2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:14.787090 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-5dcd076d-37df-4fa9-af39-ce15ea631de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:14.963094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd076d-37df-4fa9-af39-ce15ea631de2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:14.992943 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:15.092520 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fac86-9d2d-42ef-a9f2-c6c595e46a02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:15.093152 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fac86-9d2d-42ef-a9f2-c6c595e46a02 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:15.093431 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fac86-9d2d-42ef-a9f2-c6c595e46a02 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:15.093703 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fac86-9d2d-42ef-a9f2-c6c595e46a02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:15.095388 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-3e5fac86-9d2d-42ef-a9f2-c6c595e46a02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:15.189895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fac86-9d2d-42ef-a9f2-c6c595e46a02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:15.191401 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fac86-9d2d-42ef-a9f2-c6c595e46a02 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:15.210508 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 714/1440] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:17.139256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ab739-23b4-46c1-9c3f-873e36716b57 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:17.139795 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ab739-23b4-46c1-9c3f-873e36716b57 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:17.139894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ab739-23b4-46c1-9c3f-873e36716b57 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:17.141226 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 18:44:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:17.146401 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7babc5-2f90-4a65-bb91-c08c37e4992b None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:17.146881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7babc5-2f90-4a65-bb91-c08c37e4992b None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:17.146987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7babc5-2f90-4a65-bb91-c08c37e4992b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:17.188045 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:17.214125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-dc7babc5-2f90-4a65-bb91-c08c37e4992b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:17.215883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-dc7babc5-2f90-4a65-bb91-c08c37e4992b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:17.313319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 18:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:17.352117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ab739-23b4-46c1-9c3f-873e36716b57 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:17.353695 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a48ab739-23b4-46c1-9c3f-873e36716b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:17.459990 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ab739-23b4-46c1-9c3f-873e36716b57 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:17.492980 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:17.535128 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca34dd-b23f-42be-bcb0-5a96208efdbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:17.535930 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca34dd-b23f-42be-bcb0-5a96208efdbb service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:17.536271 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca34dd-b23f-42be-bcb0-5a96208efdbb service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:17.536708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca34dd-b23f-42be-bcb0-5a96208efdbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:17.538332 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-bcca34dd-b23f-42be-bcb0-5a96208efdbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:17.651504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca34dd-b23f-42be-bcb0-5a96208efdbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:17.653762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca34dd-b23f-42be-bcb0-5a96208efdbb service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:17.672997 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 728/1443] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:19.914491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab346e5-5aa4-4163-9c9b-aa26566e126b None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:19.915370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab346e5-5aa4-4163-9c9b-aa26566e126b None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:19.915370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab346e5-5aa4-4163-9c9b-aa26566e126b None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:19.917346 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 18:44:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:19.923771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de0ae5-53cd-42ed-b532-70a66998f7ad None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:19.924272 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de0ae5-53cd-42ed-b532-70a66998f7ad None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:19.924408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de0ae5-53cd-42ed-b532-70a66998f7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:19.994550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d9de0ae5-53cd-42ed-b532-70a66998f7ad None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:19.995804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d9de0ae5-53cd-42ed-b532-70a66998f7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:20.009611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:20.095185 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 18:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:20.133689 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab346e5-5aa4-4163-9c9b-aa26566e126b None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:20.135236 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-fab346e5-5aa4-4163-9c9b-aa26566e126b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:20.241297 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab346e5-5aa4-4163-9c9b-aa26566e126b None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:20.260270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:20.328080 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be409a-e557-4675-9d9d-df1f9522fd87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:20.328739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be409a-e557-4675-9d9d-df1f9522fd87 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:20.329025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be409a-e557-4675-9d9d-df1f9522fd87 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:20.329308 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be409a-e557-4675-9d9d-df1f9522fd87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:20.331002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c0be409a-e557-4675-9d9d-df1f9522fd87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:20.441079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be409a-e557-4675-9d9d-df1f9522fd87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:20.442521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be409a-e557-4675-9d9d-df1f9522fd87 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:20.461459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 717/1446] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:22.858205 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f32b71-bf17-4f12-a50a-bb1bbab967a5 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:22.858790 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f32b71-bf17-4f12-a50a-bb1bbab967a5 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:22.858790 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f32b71-bf17-4f12-a50a-bb1bbab967a5 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:22.860093 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 18:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:22.865301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94036b21-f8b7-437d-b514-98b73ce42cb0 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:22.865722 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94036b21-f8b7-437d-b514-98b73ce42cb0 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:22.865890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94036b21-f8b7-437d-b514-98b73ce42cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:22.915252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:22.936017 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-94036b21-f8b7-437d-b514-98b73ce42cb0 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:22.937268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-94036b21-f8b7-437d-b514-98b73ce42cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:23.033711 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 18:44:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:23.073439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f32b71-bf17-4f12-a50a-bb1bbab967a5 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:23.075562 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-25f32b71-bf17-4f12-a50a-bb1bbab967a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:23.179665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f32b71-bf17-4f12-a50a-bb1bbab967a5 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:23.221100 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:23.256256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e096d4-894b-4646-9ab1-9236ace550da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:23.256901 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e096d4-894b-4646-9ab1-9236ace550da service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:23.257178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e096d4-894b-4646-9ab1-9236ace550da service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:23.257447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e096d4-894b-4646-9ab1-9236ace550da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:23.259070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b3e096d4-894b-4646-9ab1-9236ace550da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:23.371592 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e096d4-894b-4646-9ab1-9236ace550da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:23.373406 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e096d4-894b-4646-9ab1-9236ace550da service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:23.394390 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 731/1449] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:44:25.561373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f792c9-e8f5-4ea3-b7b5-1217b6eec0ee None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:25.562004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f792c9-e8f5-4ea3-b7b5-1217b6eec0ee None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:25.562004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f792c9-e8f5-4ea3-b7b5-1217b6eec0ee None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:25.563230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 18:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:25.568561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0040f5-fa2f-470b-a782-5295fdd3867c None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:25.568991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0040f5-fa2f-470b-a782-5295fdd3867c None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:25.569114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0040f5-fa2f-470b-a782-5295fdd3867c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:25.637156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-0e0040f5-fa2f-470b-a782-5295fdd3867c None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:25.638369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0e0040f5-fa2f-470b-a782-5295fdd3867c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:25.659682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:25.736105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 18:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:25.776065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f792c9-e8f5-4ea3-b7b5-1217b6eec0ee None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:25.778273 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-82f792c9-e8f5-4ea3-b7b5-1217b6eec0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:25.882793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f792c9-e8f5-4ea3-b7b5-1217b6eec0ee None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:25.903328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:25.958357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e805d3-9034-4062-8c52-9e6334b56ead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:25.958976 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e805d3-9034-4062-8c52-9e6334b56ead service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:25.959254 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e805d3-9034-4062-8c52-9e6334b56ead service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:25.959527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e805d3-9034-4062-8c52-9e6334b56ead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:25.961174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-29e805d3-9034-4062-8c52-9e6334b56ead service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:26.070993 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e805d3-9034-4062-8c52-9e6334b56ead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:26.072466 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e805d3-9034-4062-8c52-9e6334b56ead service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:26.092349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 720/1452] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:28.172048 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32f3a1-bb75-45ca-9273-b76c9cf4c5c0 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:28.172628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32f3a1-bb75-45ca-9273-b76c9cf4c5c0 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:28.172628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32f3a1-bb75-45ca-9273-b76c9cf4c5c0 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:28.174025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 18:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:28.179384 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e7395-c57a-4427-9c3e-e6ea51782aab None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:28.179777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e7395-c57a-4427-9c3e-e6ea51782aab None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:28.179936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e7395-c57a-4427-9c3e-e6ea51782aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:28.244707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:28.249185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-896e7395-c57a-4427-9c3e-e6ea51782aab None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:28.250818 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-896e7395-c57a-4427-9c3e-e6ea51782aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:28.350205 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 18:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:28.388876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32f3a1-bb75-45ca-9273-b76c9cf4c5c0 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:28.390416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1e32f3a1-bb75-45ca-9273-b76c9cf4c5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:28.495525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32f3a1-bb75-45ca-9273-b76c9cf4c5c0 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:28.546320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:28.570810 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f77109f-75c4-4aee-a92e-067f48f28c0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:28.571432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77109f-75c4-4aee-a92e-067f48f28c0a service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:28.571714 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77109f-75c4-4aee-a92e-067f48f28c0a service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:28.571989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77109f-75c4-4aee-a92e-067f48f28c0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:28.575717 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1f77109f-75c4-4aee-a92e-067f48f28c0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:28.686908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f77109f-75c4-4aee-a92e-067f48f28c0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:28.688384 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f77109f-75c4-4aee-a92e-067f48f28c0a service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:28.707167 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 734/1455] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:30.807016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac16a1-8b6b-499c-b026-542780bf7ae8 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:30.807561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac16a1-8b6b-499c-b026-542780bf7ae8 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:30.807561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac16a1-8b6b-499c-b026-542780bf7ae8 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:30.809283 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 18:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:30.814315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb9783-89a6-49c3-8c0a-abf79ea4d120 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:30.814729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb9783-89a6-49c3-8c0a-abf79ea4d120 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:30.814868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb9783-89a6-49c3-8c0a-abf79ea4d120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:30.882179 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-88eb9783-89a6-49c3-8c0a-abf79ea4d120 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:30.883382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-88eb9783-89a6-49c3-8c0a-abf79ea4d120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:30.912519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:30.982100 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 18:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:31.021203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac16a1-8b6b-499c-b026-542780bf7ae8 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:31.022785 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-43ac16a1-8b6b-499c-b026-542780bf7ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:31.126834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac16a1-8b6b-499c-b026-542780bf7ae8 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:31.158580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:31.208491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d9661-62eb-46ec-84d2-61ee74cf29ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:31.209123 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d9661-62eb-46ec-84d2-61ee74cf29ee service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:31.209404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d9661-62eb-46ec-84d2-61ee74cf29ee service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:31.209676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d9661-62eb-46ec-84d2-61ee74cf29ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:31.211314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-090d9661-62eb-46ec-84d2-61ee74cf29ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:31.322134 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d9661-62eb-46ec-84d2-61ee74cf29ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:31.323583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d9661-62eb-46ec-84d2-61ee74cf29ee service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:31.343628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 723/1458] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:33.451783 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2dce2-674a-43ed-9d91-94e2ad138916 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:33.452377 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2dce2-674a-43ed-9d91-94e2ad138916 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:33.452377 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2dce2-674a-43ed-9d91-94e2ad138916 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:33.453718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 18:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:33.458938 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835b58e-e0b2-4512-ab4a-5360303de9a6 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:33.459351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835b58e-e0b2-4512-ab4a-5360303de9a6 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:33.459521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835b58e-e0b2-4512-ab4a-5360303de9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:33.527633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f835b58e-e0b2-4512-ab4a-5360303de9a6 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:33.528956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f835b58e-e0b2-4512-ab4a-5360303de9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:33.538167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:33.628558 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 18:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:44:33.666147 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2dce2-674a-43ed-9d91-94e2ad138916 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:33.667745 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-39d2dce2-674a-43ed-9d91-94e2ad138916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:33.770834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2dce2-674a-43ed-9d91-94e2ad138916 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:33.825878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:33.844397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0147ffa-abb6-4445-bf46-efd87b1fd407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:33.845017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0147ffa-abb6-4445-bf46-efd87b1fd407 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:33.845298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0147ffa-abb6-4445-bf46-efd87b1fd407 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:33.845574 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0147ffa-abb6-4445-bf46-efd87b1fd407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:33.847210 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e0147ffa-abb6-4445-bf46-efd87b1fd407 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:33.959713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0147ffa-abb6-4445-bf46-efd87b1fd407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:33.961198 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0147ffa-abb6-4445-bf46-efd87b1fd407 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:33.979467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 737/1461] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:36.031238 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27634f3-6a7e-429a-916d-358d3f0073df None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:36.031647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27634f3-6a7e-429a-916d-358d3f0073df None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:36.031863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27634f3-6a7e-429a-916d-358d3f0073df None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:36.033213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 18:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:36.038556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9560a74-c478-44db-aa9b-12c84c109f3c None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:36.038941 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9560a74-c478-44db-aa9b-12c84c109f3c None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:36.039140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9560a74-c478-44db-aa9b-12c84c109f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:36.108413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-f9560a74-c478-44db-aa9b-12c84c109f3c None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:36.109662 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f9560a74-c478-44db-aa9b-12c84c109f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:36.149871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:36.221254 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 18:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:36.258825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27634f3-6a7e-429a-916d-358d3f0073df None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:36.260409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b27634f3-6a7e-429a-916d-358d3f0073df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:36.365655 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27634f3-6a7e-429a-916d-358d3f0073df None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:36.405369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:36.440731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad9b82-c3fc-404a-b22b-b11fc2fd6699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:36.441346 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad9b82-c3fc-404a-b22b-b11fc2fd6699 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:36.441624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad9b82-c3fc-404a-b22b-b11fc2fd6699 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:36.441896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad9b82-c3fc-404a-b22b-b11fc2fd6699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:36.443600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-efad9b82-c3fc-404a-b22b-b11fc2fd6699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:36.556779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad9b82-c3fc-404a-b22b-b11fc2fd6699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:36.558287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad9b82-c3fc-404a-b22b-b11fc2fd6699 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:36.577061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 726/1464] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:38.625245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819868b-74f8-4666-ae47-1aaa4acf5bdd None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:38.626329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819868b-74f8-4666-ae47-1aaa4acf5bdd None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:38.626490 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819868b-74f8-4666-ae47-1aaa4acf5bdd None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:38.627872 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 18:44:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:38.633160 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3058bbe-d0f4-4a62-9f96-f4a82af97cb3 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:38.633595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3058bbe-d0f4-4a62-9f96-f4a82af97cb3 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:38.633721 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3058bbe-d0f4-4a62-9f96-f4a82af97cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:38.700004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-a3058bbe-d0f4-4a62-9f96-f4a82af97cb3 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:38.701336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a3058bbe-d0f4-4a62-9f96-f4a82af97cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:38.719644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:38.801512 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 18:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:44:38.837984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819868b-74f8-4666-ae47-1aaa4acf5bdd None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:38.840455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e819868b-74f8-4666-ae47-1aaa4acf5bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:38.952820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819868b-74f8-4666-ae47-1aaa4acf5bdd None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:39.019689 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:39.027955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a931b8-9d62-4b7d-a6e5-841073f8d25a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:39.028702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a931b8-9d62-4b7d-a6e5-841073f8d25a service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:39.028991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a931b8-9d62-4b7d-a6e5-841073f8d25a service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:39.029270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a931b8-9d62-4b7d-a6e5-841073f8d25a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:39.030928 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-80a931b8-9d62-4b7d-a6e5-841073f8d25a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:39.140871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a931b8-9d62-4b7d-a6e5-841073f8d25a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:39.142312 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a931b8-9d62-4b7d-a6e5-841073f8d25a service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:39.161836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 740/1467] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:41.335736 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8b141-8e64-4943-b4e1-1782443c72da None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:41.336363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8b141-8e64-4943-b4e1-1782443c72da None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:41.336363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8b141-8e64-4943-b4e1-1782443c72da None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:41.337684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 18:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:44:41.343353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fa2fe-d270-4381-9418-aaadc81a8da5 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:41.343787 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fa2fe-d270-4381-9418-aaadc81a8da5 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:41.343887 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fa2fe-d270-4381-9418-aaadc81a8da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:41.410787 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-c84fa2fe-d270-4381-9418-aaadc81a8da5 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:44:41.411990 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c84fa2fe-d270-4381-9418-aaadc81a8da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:41.461713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:41.511842 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 18:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:44:41.553493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8b141-8e64-4943-b4e1-1782443c72da None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:44:41.555034 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-eaa8b141-8e64-4943-b4e1-1782443c72da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:41.659670 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8b141-8e64-4943-b4e1-1782443c72da None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:44:41.708088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:41.734186 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94afae4-29ff-488d-bdad-f854eaffeb58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:44:41.734758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94afae4-29ff-488d-bdad-f854eaffeb58 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:44:41.735038 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94afae4-29ff-488d-bdad-f854eaffeb58 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:44:41.735308 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94afae4-29ff-488d-bdad-f854eaffeb58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:44:41.736985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f94afae4-29ff-488d-bdad-f854eaffeb58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:44:41.848947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94afae4-29ff-488d-bdad-f854eaffeb58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:44:41.850390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94afae4-29ff-488d-bdad-f854eaffeb58 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:44:41.869197 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 729/1470] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:44:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:44:51.509706 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=22 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:44:51.866975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=66 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:45:01.565869 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e04397-4b4d-4523-b4f7-4bcf6b49a413 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:01.566434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e04397-4b4d-4523-b4f7-4bcf6b49a413 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:01.566434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e04397-4b4d-4523-b4f7-4bcf6b49a413 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:01.567758 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 742/1471] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:45:01.572981 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30432dd6-c759-455d-948e-3e3635fdbb5a None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:01.573657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30432dd6-c759-455d-948e-3e3635fdbb5a None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:01.573657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30432dd6-c759-455d-948e-3e3635fdbb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:01.661302 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-30432dd6-c759-455d-948e-3e3635fdbb5a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:45:01.662505 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-30432dd6-c759-455d-948e-3e3635fdbb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:01.761077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 730/1472] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:45:01.844538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b23a4e-3777-4b30-8eed-3ea5e1bf7c4d None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:01.844828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b23a4e-3777-4b30-8eed-3ea5e1bf7c4d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:01.845110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b23a4e-3777-4b30-8eed-3ea5e1bf7c4d None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:01.846453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 743/1473] 10.222.0.69 () {54 vars in 966 bytes} [Wed Aug 3 18:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:45:01.851200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a3daa-be82-48ce-88b5-132c9be7fb14 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:01.851494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a3daa-be82-48ce-88b5-132c9be7fb14 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:01.851773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a3daa-be82-48ce-88b5-132c9be7fb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:01.929157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-835a3daa-be82-48ce-88b5-132c9be7fb14 None None] MFA Rules not processed for user `4799b0c9e5aa47d389121d2183666d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:45:01.930384 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-835a3daa-be82-48ce-88b5-132c9be7fb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:01.953338 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:45:02.012049 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 731/1474] 10.222.0.69 () {58 vars in 1063 bytes} [Wed Aug 3 18:45:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:45:02.020039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b23a4e-3777-4b30-8eed-3ea5e1bf7c4d None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:45:02.021602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-16b23a4e-3777-4b30-8eed-3ea5e1bf7c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:02.127542 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b23a4e-3777-4b30-8eed-3ea5e1bf7c4d None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:45:02.202354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4b245-7ff4-456e-9751-bc7a4ca4e5a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4799b0c9e5aa47d389121d2183666d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:45:02.202976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4b245-7ff4-456e-9751-bc7a4ca4e5a2 service glance] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:02.203258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4b245-7ff4-456e-9751-bc7a4ca4e5a2 service glance] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:02.203534 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4b245-7ff4-456e-9751-bc7a4ca4e5a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:02.205205 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1fc4b245-7ff4-456e-9751-bc7a4ca4e5a2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:02.260661 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:45:02.316550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4b245-7ff4-456e-9751-bc7a4ca4e5a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:45:02.317999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4b245-7ff4-456e-9751-bc7a4ca4e5a2 service glance] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:45:02.337473 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 744/1475] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:45:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:45:04.471048 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6495c-b5e3-4a79-a0e9-52ac1e3db209 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:04.471622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6495c-b5e3-4a79-a0e9-52ac1e3db209 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:04.471725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6495c-b5e3-4a79-a0e9-52ac1e3db209 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:04.473221 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 732/1476] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:45:04.478272 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72667ee-b328-4ba9-8eb8-951ac0ca056f None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:04.478686 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72667ee-b328-4ba9-8eb8-951ac0ca056f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:04.478824 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72667ee-b328-4ba9-8eb8-951ac0ca056f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:04.546778 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-e72667ee-b328-4ba9-8eb8-951ac0ca056f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:45:04.547994 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e72667ee-b328-4ba9-8eb8-951ac0ca056f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:04.653504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 745/1477] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:45:04.680226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6495c-b5e3-4a79-a0e9-52ac1e3db209 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:45:04.681781 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4dd6495c-b5e3-4a79-a0e9-52ac1e3db209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:04.809950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6495c-b5e3-4a79-a0e9-52ac1e3db209 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:45:04.858377 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:45:04.904073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567b3a6-250d-419b-a422-6a8e9b5902e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:45:04.904722 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567b3a6-250d-419b-a422-6a8e9b5902e4 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:04.905004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567b3a6-250d-419b-a422-6a8e9b5902e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:04.905275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567b3a6-250d-419b-a422-6a8e9b5902e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:04.905974 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567b3a6-250d-419b-a422-6a8e9b5902e4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:45:04.907683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: WARNING py.warnings [None req-7567b3a6-250d-419b-a422-6a8e9b5902e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 18:45:04.907683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: warnings.warn(msg) Aug 03 18:45:04.907683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: Aug 03 18:45:04.908854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567b3a6-250d-419b-a422-6a8e9b5902e4 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:45:04.928518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 733/1478] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Aug 3 18:45:04 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:45:06.977817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a3817-59e3-44c4-bc79-f908e5aa1481 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:06.978397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a3817-59e3-44c4-bc79-f908e5aa1481 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:06.978397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a3817-59e3-44c4-bc79-f908e5aa1481 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:06.979734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 746/1479] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:45:06.985326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880cd05-c730-426c-99c0-ac4d5d87ba0f None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:06.985772 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880cd05-c730-426c-99c0-ac4d5d87ba0f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:06.985895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880cd05-c730-426c-99c0-ac4d5d87ba0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:07.054558 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-9880cd05-c730-426c-99c0-ac4d5d87ba0f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:45:07.055517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9880cd05-c730-426c-99c0-ac4d5d87ba0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:07.153129 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 734/1480] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:45:07.198156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a3817-59e3-44c4-bc79-f908e5aa1481 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:45:07.199833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-326a3817-59e3-44c4-bc79-f908e5aa1481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:07.209666 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:45:07.351434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a3817-59e3-44c4-bc79-f908e5aa1481 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:45:07.438484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:45:07.472142 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c62381-28a6-46af-8d42-c70d93395baa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:45:07.472791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c62381-28a6-46af-8d42-c70d93395baa service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:07.473068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c62381-28a6-46af-8d42-c70d93395baa service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:07.473333 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c62381-28a6-46af-8d42-c70d93395baa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:07.474942 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b7c62381-28a6-46af-8d42-c70d93395baa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:07.566908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c62381-28a6-46af-8d42-c70d93395baa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:45:07.568544 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c62381-28a6-46af-8d42-c70d93395baa service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:45:07.587120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 747/1481] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:45:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:45:09.829486 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09e5dd-eb36-479f-a8a3-3e14f2792be2 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:09.830359 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09e5dd-eb36-479f-a8a3-3e14f2792be2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:09.830359 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09e5dd-eb36-479f-a8a3-3e14f2792be2 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:09.831436 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 735/1482] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:45:09.836892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedeea8-d2bf-427a-b336-d0ce01c494e9 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:09.837355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedeea8-d2bf-427a-b336-d0ce01c494e9 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:09.837464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedeea8-d2bf-427a-b336-d0ce01c494e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:09.903921 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-6eedeea8-d2bf-427a-b336-d0ce01c494e9 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:45:09.905144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-6eedeea8-d2bf-427a-b336-d0ce01c494e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:09.953892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:45:10.005488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 748/1483] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:45:10.042827 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09e5dd-eb36-479f-a8a3-3e14f2792be2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:45:10.044389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-7a09e5dd-eb36-479f-a8a3-3e14f2792be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:10.068665 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:45:10.151268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09e5dd-eb36-479f-a8a3-3e14f2792be2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:45:10.224937 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04466e1e-9e81-4f5c-9054-53308f1bae0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:45:10.225561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04466e1e-9e81-4f5c-9054-53308f1bae0f service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:10.225845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04466e1e-9e81-4f5c-9054-53308f1bae0f service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:10.226121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04466e1e-9e81-4f5c-9054-53308f1bae0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:10.227750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-04466e1e-9e81-4f5c-9054-53308f1bae0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:10.303225 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:45:10.339772 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04466e1e-9e81-4f5c-9054-53308f1bae0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:45:10.341280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04466e1e-9e81-4f5c-9054-53308f1bae0f service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:45:10.359995 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 736/1484] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:45:12.449439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfab78-85d3-414b-84c7-4f007815c88e None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:12.450275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfab78-85d3-414b-84c7-4f007815c88e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:12.450275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfab78-85d3-414b-84c7-4f007815c88e None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:12.451395 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 749/1485] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:45:12.456718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b754a-aeeb-492a-be7d-7d0b6009d2bd None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:12.457149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b754a-aeeb-492a-be7d-7d0b6009d2bd None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:12.457262 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b754a-aeeb-492a-be7d-7d0b6009d2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:12.525195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f75b754a-aeeb-492a-be7d-7d0b6009d2bd None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:45:12.526408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f75b754a-aeeb-492a-be7d-7d0b6009d2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:12.625659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 737/1486] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:45:12.663850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfab78-85d3-414b-84c7-4f007815c88e None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:45:12.665449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-adbfab78-85d3-414b-84c7-4f007815c88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:12.777717 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfab78-85d3-414b-84c7-4f007815c88e None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:45:12.849631 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:45:12.851679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f3abc-eee8-4e94-a664-29d9ef7f2199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:45:12.852395 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f3abc-eee8-4e94-a664-29d9ef7f2199 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:12.852876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f3abc-eee8-4e94-a664-29d9ef7f2199 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:12.853187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f3abc-eee8-4e94-a664-29d9ef7f2199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:12.855750 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-321f3abc-eee8-4e94-a664-29d9ef7f2199 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:12.968547 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f3abc-eee8-4e94-a664-29d9ef7f2199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:45:12.969997 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f3abc-eee8-4e94-a664-29d9ef7f2199 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:45:12.988729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 750/1487] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:45:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:45:15.085055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89f719-c446-498d-a9d4-fffe32056670 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:15.085699 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89f719-c446-498d-a9d4-fffe32056670 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:15.085699 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89f719-c446-498d-a9d4-fffe32056670 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:15.086984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 738/1488] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:45:15.092355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c724056-1994-458f-8499-8dab3541dc4b None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:15.092790 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c724056-1994-458f-8499-8dab3541dc4b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:15.092896 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c724056-1994-458f-8499-8dab3541dc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:15.161609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-1c724056-1994-458f-8499-8dab3541dc4b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:45:15.162821 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1c724056-1994-458f-8499-8dab3541dc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:15.217207 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:45:15.264149 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 751/1489] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:45:15.305778 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89f719-c446-498d-a9d4-fffe32056670 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:45:15.307312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ac89f719-c446-498d-a9d4-fffe32056670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:15.343879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:45:15.413669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89f719-c446-498d-a9d4-fffe32056670 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:45:15.487697 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877da630-0c44-4089-9245-2f83ed1df6a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:45:15.488239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877da630-0c44-4089-9245-2f83ed1df6a5 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:15.488549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877da630-0c44-4089-9245-2f83ed1df6a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:15.488822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877da630-0c44-4089-9245-2f83ed1df6a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:15.490445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-877da630-0c44-4089-9245-2f83ed1df6a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:15.574410 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:45:15.604559 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877da630-0c44-4089-9245-2f83ed1df6a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:45:15.606568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877da630-0c44-4089-9245-2f83ed1df6a5 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:45:15.624866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 739/1490] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:45:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:45:18.028782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b8fd3-3a80-47b9-ba47-5a1fc20d9edc None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:18.029340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b8fd3-3a80-47b9-ba47-5a1fc20d9edc None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:18.029340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b8fd3-3a80-47b9-ba47-5a1fc20d9edc None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:18.030709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 752/1491] 10.222.0.69 () {54 vars in 926 bytes} [Wed Aug 3 18:45:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:45:18.036037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd8af9-f9d0-4960-97f3-276b7f4949e0 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:18.036476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd8af9-f9d0-4960-97f3-276b7f4949e0 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:18.036642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd8af9-f9d0-4960-97f3-276b7f4949e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:18.103059 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-21bd8af9-f9d0-4960-97f3-276b7f4949e0 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:45:18.104273 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-21bd8af9-f9d0-4960-97f3-276b7f4949e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:18.204617 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 740/1492] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Aug 3 18:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:45:18.243690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b8fd3-3a80-47b9-ba47-5a1fc20d9edc None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:45:18.245294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d43b8fd3-3a80-47b9-ba47-5a1fc20d9edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:18.351818 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b8fd3-3a80-47b9-ba47-5a1fc20d9edc None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:45:18.426871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da3b6f-36b3-4c94-83f6-db87e8fc6649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:45:18.427495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3b6f-36b3-4c94-83f6-db87e8fc6649 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:45:18.427778 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3b6f-36b3-4c94-83f6-db87e8fc6649 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:45:18.428054 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3b6f-36b3-4c94-83f6-db87e8fc6649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:45:18.429720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-20da3b6f-36b3-4c94-83f6-db87e8fc6649 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:45:18.439668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:45:18.541951 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da3b6f-36b3-4c94-83f6-db87e8fc6649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:45:18.544967 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da3b6f-36b3-4c94-83f6-db87e8fc6649 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:45:18.567284 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 753/1493] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:45:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:45:28.202776 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=82 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:45:28.564959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=50 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:46:37.026873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30437f-e03c-46d3-868a-b4c16e259229 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:37.027471 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30437f-e03c-46d3-868a-b4c16e259229 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:37.027589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30437f-e03c-46d3-868a-b4c16e259229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:37.103887 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ed30437f-e03c-46d3-868a-b4c16e259229 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:46:37.106119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ed30437f-e03c-46d3-868a-b4c16e259229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:37.206584 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 741/1494] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:46:37.215381 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdd4ca-6883-4033-9542-0dd63a273b5f None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:37.215918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdd4ca-6883-4033-9542-0dd63a273b5f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:37.215918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdd4ca-6883-4033-9542-0dd63a273b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:37.291810 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-6ebdd4ca-6883-4033-9542-0dd63a273b5f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:46:37.293606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-6ebdd4ca-6883-4033-9542-0dd63a273b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:37.391167 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 754/1495] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:46:37.395835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30437f-e03c-46d3-868a-b4c16e259229 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:46:37.397097 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ed30437f-e03c-46d3-868a-b4c16e259229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:37.409122 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30437f-e03c-46d3-868a-b4c16e259229 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:46:37.472059 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:46:37.507008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06096c68-2298-4851-8e96-ca32000182a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:46:37.507639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06096c68-2298-4851-8e96-ca32000182a8 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:37.507920 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06096c68-2298-4851-8e96-ca32000182a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:37.508197 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06096c68-2298-4851-8e96-ca32000182a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:37.508927 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06096c68-2298-4851-8e96-ca32000182a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:46:37.509233 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06096c68-2298-4851-8e96-ca32000182a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:46:37.510834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: WARNING py.warnings [None req-06096c68-2298-4851-8e96-ca32000182a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 18:46:37.510834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: warnings.warn(msg) Aug 03 18:46:37.510834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: Aug 03 18:46:37.512097 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06096c68-2298-4851-8e96-ca32000182a8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:46:37.523501 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 742/1496] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:46:37.532619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdd4ca-6883-4033-9542-0dd63a273b5f None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:46:37.541922 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdd4ca-6883-4033-9542-0dd63a273b5f None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:46:37.603975 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:46:37.638659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb78e3-64b2-475d-accc-0f8ad4bc6654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:46:37.639315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb78e3-64b2-475d-accc-0f8ad4bc6654 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:37.639600 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb78e3-64b2-475d-accc-0f8ad4bc6654 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:37.639902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb78e3-64b2-475d-accc-0f8ad4bc6654 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:37.640625 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb78e3-64b2-475d-accc-0f8ad4bc6654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:46:37.642356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: WARNING py.warnings [None req-e0eb78e3-64b2-475d-accc-0f8ad4bc6654 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 18:46:37.642356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: warnings.warn(msg) Aug 03 18:46:37.642356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: Aug 03 18:46:37.643551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb78e3-64b2-475d-accc-0f8ad4bc6654 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:46:37.702362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 755/1497] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:46:37 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:46:37.706723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06096c68-2298-4851-8e96-ca32000182a8 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:46:37.708249 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-06096c68-2298-4851-8e96-ca32000182a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:37.742116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06096c68-2298-4851-8e96-ca32000182a8 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:46:37.840323 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c23aaf-7d74-463f-a48d-734c34273d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:46:37.840960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c23aaf-7d74-463f-a48d-734c34273d1a admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:37.841243 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c23aaf-7d74-463f-a48d-734c34273d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:37.841517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c23aaf-7d74-463f-a48d-734c34273d1a admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:37.842185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c23aaf-7d74-463f-a48d-734c34273d1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:46:37.843210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: WARNING py.warnings [None req-16c23aaf-7d74-463f-a48d-734c34273d1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 18:46:37.843210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: warnings.warn(msg) Aug 03 18:46:37.843210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: Aug 03 18:46:37.844456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c23aaf-7d74-463f-a48d-734c34273d1a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:46:37.899131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 743/1498] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:46:37 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:46:37.904349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb78e3-64b2-475d-accc-0f8ad4bc6654 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:46:37.905876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e0eb78e3-64b2-475d-accc-0f8ad4bc6654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:37.970922 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb78e3-64b2-475d-accc-0f8ad4bc6654 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:46:38.053769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:46:38.066280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb7738-351c-42d5-8fd3-42ab6a329d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:46:38.066907 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb7738-351c-42d5-8fd3-42ab6a329d66 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:38.067189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb7738-351c-42d5-8fd3-42ab6a329d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:38.067465 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb7738-351c-42d5-8fd3-42ab6a329d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:38.068168 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb7738-351c-42d5-8fd3-42ab6a329d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:46:38.069305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: WARNING py.warnings [None req-8ebb7738-351c-42d5-8fd3-42ab6a329d66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 18:46:38.069305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: warnings.warn(msg) Aug 03 18:46:38.069305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: Aug 03 18:46:38.070202 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb7738-351c-42d5-8fd3-42ab6a329d66 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:46:38.083567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 756/1499] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:46:38.088203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c23aaf-7d74-463f-a48d-734c34273d1a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:46:38.089289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-16c23aaf-7d74-463f-a48d-734c34273d1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:38.100831 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c23aaf-7d74-463f-a48d-734c34273d1a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:46:38.126934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:46:38.204607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021240e-6613-402e-ae3b-0a086d2c83da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:46:38.205241 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021240e-6613-402e-ae3b-0a086d2c83da admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:38.205525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021240e-6613-402e-ae3b-0a086d2c83da admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:38.205800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021240e-6613-402e-ae3b-0a086d2c83da admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:38.206502 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021240e-6613-402e-ae3b-0a086d2c83da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:46:38.207512 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: WARNING py.warnings [None req-d021240e-6613-402e-ae3b-0a086d2c83da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 18:46:38.207512 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: warnings.warn(msg) Aug 03 18:46:38.207512 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: Aug 03 18:46:38.208430 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021240e-6613-402e-ae3b-0a086d2c83da admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:46:38.220608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 744/1500] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:46:38.224830 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb7738-351c-42d5-8fd3-42ab6a329d66 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:46:38.226388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8ebb7738-351c-42d5-8fd3-42ab6a329d66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:38.261140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb7738-351c-42d5-8fd3-42ab6a329d66 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:46:38.358436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db452a-3484-4fc9-bae7-61203381f002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:46:38.359036 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db452a-3484-4fc9-bae7-61203381f002 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:38.359321 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db452a-3484-4fc9-bae7-61203381f002 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:38.359599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db452a-3484-4fc9-bae7-61203381f002 admin admin] PATH_INFO: `/v3/projects/17a2e5fddaf242aaa7577860de6de147/users/5b1fdd48d6a841e291c263a6c0a5737f/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:38.383136 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db452a-3484-4fc9-bae7-61203381f002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a2e5fddaf242aaa7577860de6de147, user_id=5b1fdd48d6a841e291c263a6c0a5737f, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:46:38.384172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: WARNING py.warnings [None req-c4db452a-3484-4fc9-bae7-61203381f002 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 18:46:38.384172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: warnings.warn(msg) Aug 03 18:46:38.384172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: Aug 03 18:46:38.386956 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db452a-3484-4fc9-bae7-61203381f002 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:46:38.411237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 757/1501] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:46:38 2022] PUT /identity/v3/projects/17a2e5fddaf242aaa7577860de6de147/users/5b1fdd48d6a841e291c263a6c0a5737f/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:46:38.423874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ade0e-1498-4494-8fb4-5254c93a6bb6 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:38.424171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ade0e-1498-4494-8fb4-5254c93a6bb6 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:38.424481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ade0e-1498-4494-8fb4-5254c93a6bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:38.482162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-732ade0e-1498-4494-8fb4-5254c93a6bb6 None None] MFA Rules not processed for user `5b1fdd48d6a841e291c263a6c0a5737f`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:46:38.483366 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-732ade0e-1498-4494-8fb4-5254c93a6bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:38.561528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 745/1502] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:46:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:46:38.575152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece9f2e-6edd-4724-b6a3-24b0fc5dafe0 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:38.575441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece9f2e-6edd-4724-b6a3-24b0fc5dafe0 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:38.575716 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece9f2e-6edd-4724-b6a3-24b0fc5dafe0 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:38.577109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 758/1503] 10.222.0.69 () {54 vars in 964 bytes} [Wed Aug 3 18:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:46:38.581923 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68fb3f-39a3-4a5c-a538-247574db4805 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:38.582219 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68fb3f-39a3-4a5c-a538-247574db4805 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:38.582530 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68fb3f-39a3-4a5c-a538-247574db4805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:38.615346 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:46:38.660020 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-be68fb3f-39a3-4a5c-a538-247574db4805 None None] MFA Rules not processed for user `c7d0fde945184c93b8fd91d15b66f534`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:46:38.661279 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-be68fb3f-39a3-4a5c-a538-247574db4805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:38.739175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 746/1504] 10.222.0.69 () {58 vars in 1061 bytes} [Wed Aug 3 18:46:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:46:38.747535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece9f2e-6edd-4724-b6a3-24b0fc5dafe0 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:46:38.749147 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-eece9f2e-6edd-4724-b6a3-24b0fc5dafe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:38.800508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:46:38.854999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece9f2e-6edd-4724-b6a3-24b0fc5dafe0 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:46:38.930945 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6fb34-4106-4c1f-b53e-375e333c32c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:46:38.931519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6fb34-4106-4c1f-b53e-375e333c32c4 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:38.931821 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6fb34-4106-4c1f-b53e-375e333c32c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:38.932100 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6fb34-4106-4c1f-b53e-375e333c32c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:38.933831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-41a6fb34-4106-4c1f-b53e-375e333c32c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:39.033281 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6fb34-4106-4c1f-b53e-375e333c32c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:46:39.034755 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6fb34-4106-4c1f-b53e-375e333c32c4 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:46:39.038266 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:46:39.054453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 759/1505] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:46:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:46:39.099520 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7f969-256d-4fe7-b655-1acb3e1dcc4f None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:39.099916 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7f969-256d-4fe7-b655-1acb3e1dcc4f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:39.100082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7f969-256d-4fe7-b655-1acb3e1dcc4f None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:39.101461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 747/1506] 10.222.0.69 () {52 vars in 830 bytes} [Wed Aug 3 18:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:46:39.105754 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6fb34-4106-4c1f-b53e-375e333c32c4 service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:46:39.107005 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-41a6fb34-4106-4c1f-b53e-375e333c32c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:39.209616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6fb34-4106-4c1f-b53e-375e333c32c4 service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:46:39.324956 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e73edf-8b80-44bd-b24a-a6d8d2b83dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:46:39.325680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e73edf-8b80-44bd-b24a-a6d8d2b83dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:39.325886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e73edf-8b80-44bd-b24a-a6d8d2b83dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:39.326171 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e73edf-8b80-44bd-b24a-a6d8d2b83dbd admin admin] PATH_INFO: `/v3/users/5b1fdd48d6a841e291c263a6c0a5737f` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:39.341404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e73edf-8b80-44bd-b24a-a6d8d2b83dbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b1fdd48d6a841e291c263a6c0a5737f)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:46:39.342460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: WARNING py.warnings [None req-15e73edf-8b80-44bd-b24a-a6d8d2b83dbd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 18:46:39.342460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: warnings.warn(msg) Aug 03 18:46:39.342460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: Aug 03 18:46:39.343697 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e73edf-8b80-44bd-b24a-a6d8d2b83dbd admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:46:39.356957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:46:39.416528 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-15e73edf-8b80-44bd-b24a-a6d8d2b83dbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b1fdd48d6a841e291c263a6c0a5737f'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:46:39.426512 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-15e73edf-8b80-44bd-b24a-a6d8d2b83dbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b1fdd48d6a841e291c263a6c0a5737f'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:46:39.451645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-15e73edf-8b80-44bd-b24a-a6d8d2b83dbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b1fdd48d6a841e291c263a6c0a5737f'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:46:39.465853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-15e73edf-8b80-44bd-b24a-a6d8d2b83dbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b1fdd48d6a841e291c263a6c0a5737f'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:46:39.466366 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-15e73edf-8b80-44bd-b24a-a6d8d2b83dbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b1fdd48d6a841e291c263a6c0a5737f'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:46:39.471873 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 760/1507] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:46:39 2022] DELETE /identity/v3/users/5b1fdd48d6a841e291c263a6c0a5737f => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:46:39.476300 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4cad8-9187-4793-9a2f-08065414ca43 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:39.476602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4cad8-9187-4793-9a2f-08065414ca43 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:39.476883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4cad8-9187-4793-9a2f-08065414ca43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:39.535809 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-17e4cad8-9187-4793-9a2f-08065414ca43 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:46:39.537033 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-17e4cad8-9187-4793-9a2f-08065414ca43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:39.612974 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:46:39.626902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 748/1508] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:46:39.639749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e73edf-8b80-44bd-b24a-a6d8d2b83dbd admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:46:39.641040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-15e73edf-8b80-44bd-b24a-a6d8d2b83dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:39.746387 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e73edf-8b80-44bd-b24a-a6d8d2b83dbd admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:46:39.802235 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:46:39.823737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151a82f-8982-4c5d-a89c-f6693844856b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:46:39.824386 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151a82f-8982-4c5d-a89c-f6693844856b service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:39.824672 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151a82f-8982-4c5d-a89c-f6693844856b service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:39.824949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151a82f-8982-4c5d-a89c-f6693844856b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:39.826592 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-9151a82f-8982-4c5d-a89c-f6693844856b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:39.935041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151a82f-8982-4c5d-a89c-f6693844856b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:46:39.936525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151a82f-8982-4c5d-a89c-f6693844856b service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:46:39.955329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 761/1509] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:46:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:46:40.294146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4cad8-9187-4793-9a2f-08065414ca43 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:46:40.295403 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-17e4cad8-9187-4793-9a2f-08065414ca43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:46:40.307068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4cad8-9187-4793-9a2f-08065414ca43 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:46:40.410174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf36936-3b20-496b-9ea2-b4668febb60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:46:40.410822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf36936-3b20-496b-9ea2-b4668febb60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:46:40.411107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf36936-3b20-496b-9ea2-b4668febb60b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:46:40.411380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf36936-3b20-496b-9ea2-b4668febb60b admin admin] PATH_INFO: `/v3/projects/17a2e5fddaf242aaa7577860de6de147` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:46:40.422385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf36936-3b20-496b-9ea2-b4668febb60b admin admin] RBAC: Authorizing `identity:delete_project(project_id=17a2e5fddaf242aaa7577860de6de147)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:46:40.423406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: WARNING py.warnings [None req-cdf36936-3b20-496b-9ea2-b4668febb60b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 18:46:40.423406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: warnings.warn(msg) Aug 03 18:46:40.423406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: Aug 03 18:46:40.424647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf36936-3b20-496b-9ea2-b4668febb60b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:46:40.482305 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-cdf36936-3b20-496b-9ea2-b4668febb60b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17a2e5fddaf242aaa7577860de6de147', 'request_id': 'req-cdf36936-3b20-496b-9ea2-b4668febb60b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:46:40.488321 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-cdf36936-3b20-496b-9ea2-b4668febb60b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17a2e5fddaf242aaa7577860de6de147', 'request_id': 'req-cdf36936-3b20-496b-9ea2-b4668febb60b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:46:40.501165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-cdf36936-3b20-496b-9ea2-b4668febb60b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17a2e5fddaf242aaa7577860de6de147', 'request_id': 'req-cdf36936-3b20-496b-9ea2-b4668febb60b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:46:40.527509 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-cdf36936-3b20-496b-9ea2-b4668febb60b admin admin] The token cache is being invalidate because project 17a2e5fddaf242aaa7577860de6de147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:46:40.527796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-cdf36936-3b20-496b-9ea2-b4668febb60b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:46:40.528346 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-cdf36936-3b20-496b-9ea2-b4668febb60b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:46:40.529695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 749/1510] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:46:40 2022] DELETE /identity/v3/projects/17a2e5fddaf242aaa7577860de6de147 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:46:49.955682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=62 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:46:50.524807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:48:15.592019 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabef3a-4b6c-41d9-9786-4352344074b0 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:15.592685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabef3a-4b6c-41d9-9786-4352344074b0 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:15.592685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabef3a-4b6c-41d9-9786-4352344074b0 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:15.593991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 18:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:48:15.599066 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fe753-82cb-4e9f-9fcb-223c09bf4cf6 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:15.599499 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fe753-82cb-4e9f-9fcb-223c09bf4cf6 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:15.599638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fe753-82cb-4e9f-9fcb-223c09bf4cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:15.683887 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-d13fe753-82cb-4e9f-9fcb-223c09bf4cf6 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:48:15.685145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d13fe753-82cb-4e9f-9fcb-223c09bf4cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:48:15.792454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 18:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:48:15.947828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabef3a-4b6c-41d9-9786-4352344074b0 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:48:15.950073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7eabef3a-4b6c-41d9-9786-4352344074b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:48:16.076179 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabef3a-4b6c-41d9-9786-4352344074b0 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:48:16.081787 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8a24c-3b01-4bf8-afb7-ebc7a341658f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:48:16.082403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8a24c-3b01-4bf8-afb7-ebc7a341658f service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:16.082684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8a24c-3b01-4bf8-afb7-ebc7a341658f service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:16.082960 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8a24c-3b01-4bf8-afb7-ebc7a341658f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:16.092262 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8a24c-3b01-4bf8-afb7-ebc7a341658f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:48:16.093895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8a24c-3b01-4bf8-afb7-ebc7a341658f service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:48:16.104072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 763/1513] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:48:16.942562 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297aa77-7be2-412c-9fe1-aa694c4b29ad None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:16.943053 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297aa77-7be2-412c-9fe1-aa694c4b29ad None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:16.943159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297aa77-7be2-412c-9fe1-aa694c4b29ad None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:16.945006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 18:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:48:16.949543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b9851-abe6-45cb-94e4-066a443de522 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:16.949957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b9851-abe6-45cb-94e4-066a443de522 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:16.950115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b9851-abe6-45cb-94e4-066a443de522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:16.969731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-2e9b9851-abe6-45cb-94e4-066a443de522 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:48:16.970938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2e9b9851-abe6-45cb-94e4-066a443de522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:48:16.979539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 18:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:48:17.131252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297aa77-7be2-412c-9fe1-aa694c4b29ad None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:48:17.146161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297aa77-7be2-412c-9fe1-aa694c4b29ad None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:48:17.152267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e838f9-fd19-4ca5-8475-a14584aa6feb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:48:17.152905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e838f9-fd19-4ca5-8475-a14584aa6feb service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:17.153193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e838f9-fd19-4ca5-8475-a14584aa6feb service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:17.153472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e838f9-fd19-4ca5-8475-a14584aa6feb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:17.161623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e838f9-fd19-4ca5-8475-a14584aa6feb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:48:17.163851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e838f9-fd19-4ca5-8475-a14584aa6feb service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:48:17.173421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 752/1516] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:48:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:48:17.346541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a62e0-6573-45a2-adbc-b7730a1506be None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:17.346834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a62e0-6573-45a2-adbc-b7730a1506be None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:17.347118 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a62e0-6573-45a2-adbc-b7730a1506be None None] PATH_INFO: `/v3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:17.348492 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 765/1517] 10.222.0.73 () {54 vars in 925 bytes} [Wed Aug 3 18:48:17 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:48:17.354669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c15dc-136b-4758-bcaf-a384fcef55cd None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:17.354962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c15dc-136b-4758-bcaf-a384fcef55cd None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:17.355246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c15dc-136b-4758-bcaf-a384fcef55cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:17.417109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-8c8c15dc-136b-4758-bcaf-a384fcef55cd None None] MFA Rules not processed for user `fea93ceb4cbc49c2b4b280a4ddc257b7`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:48:17.418329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8c8c15dc-136b-4758-bcaf-a384fcef55cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:48:17.435292 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:48:17.490913 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 753/1518] 10.222.0.73 () {58 vars in 1015 bytes} [Wed Aug 3 18:48:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:48:26.762723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c6098-b3b3-43b2-8fff-410836775ede None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:26.763255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c6098-b3b3-43b2-8fff-410836775ede None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:26.763255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c6098-b3b3-43b2-8fff-410836775ede None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:26.764565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 766/1519] 10.222.0.69 () {54 vars in 961 bytes} [Wed Aug 3 18:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:48:26.769489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7be50-e6d2-4e8f-9621-1b129629ca30 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:26.769955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7be50-e6d2-4e8f-9621-1b129629ca30 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:26.770055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7be50-e6d2-4e8f-9621-1b129629ca30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:26.789927 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f0e7be50-e6d2-4e8f-9621-1b129629ca30 None None] MFA Rules not processed for user `fea93ceb4cbc49c2b4b280a4ddc257b7`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:48:26.791727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f0e7be50-e6d2-4e8f-9621-1b129629ca30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:48:26.800588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 754/1520] 10.222.0.69 () {58 vars in 1058 bytes} [Wed Aug 3 18:48:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:48:26.808301 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c6098-b3b3-43b2-8fff-410836775ede None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:48:26.821514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c6098-b3b3-43b2-8fff-410836775ede None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:48:26.826622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785e470-40ae-421c-a5b6-db5030b26c38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:48:26.827220 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785e470-40ae-421c-a5b6-db5030b26c38 service placement] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:26.827503 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785e470-40ae-421c-a5b6-db5030b26c38 service placement] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:26.827780 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785e470-40ae-421c-a5b6-db5030b26c38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:26.835886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785e470-40ae-421c-a5b6-db5030b26c38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:48:26.837501 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785e470-40ae-421c-a5b6-db5030b26c38 service placement] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:48:26.847870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 767/1521] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:48:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:48:28.418489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52ca9b-c441-4b49-a14c-38f52faf603f None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:28.419069 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52ca9b-c441-4b49-a14c-38f52faf603f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:28.419069 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52ca9b-c441-4b49-a14c-38f52faf603f None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:28.420438 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 18:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:48:28.425322 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b47d97-1cd0-40a6-bd12-71dd10d75c50 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:28.425739 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b47d97-1cd0-40a6-bd12-71dd10d75c50 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:28.425935 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b47d97-1cd0-40a6-bd12-71dd10d75c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:28.445850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-f9b47d97-1cd0-40a6-bd12-71dd10d75c50 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:48:28.447044 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f9b47d97-1cd0-40a6-bd12-71dd10d75c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:48:28.454884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 18:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:48:28.611347 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52ca9b-c441-4b49-a14c-38f52faf603f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:48:28.626250 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52ca9b-c441-4b49-a14c-38f52faf603f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:48:28.632554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db82f6a-2fad-43c2-a4db-8589aa9898d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:48:28.633175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db82f6a-2fad-43c2-a4db-8589aa9898d8 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:28.633456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db82f6a-2fad-43c2-a4db-8589aa9898d8 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:28.633730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db82f6a-2fad-43c2-a4db-8589aa9898d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:28.642499 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db82f6a-2fad-43c2-a4db-8589aa9898d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:48:28.644076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db82f6a-2fad-43c2-a4db-8589aa9898d8 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:48:28.653635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 756/1524] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:48:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:48:29.498870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de051fc-e0c1-4f97-9fec-9634e80f1070 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:29.499471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de051fc-e0c1-4f97-9fec-9634e80f1070 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:29.499471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de051fc-e0c1-4f97-9fec-9634e80f1070 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:29.500809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 18:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:48:29.505867 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f13bf-2364-4883-b709-f527663157b9 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:29.506292 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f13bf-2364-4883-b709-f527663157b9 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:29.506425 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f13bf-2364-4883-b709-f527663157b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:29.526304 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-e23f13bf-2364-4883-b709-f527663157b9 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:48:29.527556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e23f13bf-2364-4883-b709-f527663157b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:48:29.536037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 18:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:48:29.689536 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de051fc-e0c1-4f97-9fec-9634e80f1070 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:48:29.705320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de051fc-e0c1-4f97-9fec-9634e80f1070 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:48:29.710936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a33a80-0772-4eb2-9f25-c852f723e361 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:48:29.711643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a33a80-0772-4eb2-9f25-c852f723e361 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:29.711838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a33a80-0772-4eb2-9f25-c852f723e361 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:29.712173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a33a80-0772-4eb2-9f25-c852f723e361 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:29.723187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a33a80-0772-4eb2-9f25-c852f723e361 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:48:29.724820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a33a80-0772-4eb2-9f25-c852f723e361 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:48:29.735621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 770/1527] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:48:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:48:33.717843 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6f6df-dd01-48f4-81fe-68199752a298 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:33.718445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6f6df-dd01-48f4-81fe-68199752a298 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:33.718445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6f6df-dd01-48f4-81fe-68199752a298 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:33.719796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 18:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:48:33.724938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd0a47-1d5b-4d9a-bc3f-0801ea7c72db None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:33.725328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd0a47-1d5b-4d9a-bc3f-0801ea7c72db None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:33.725488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd0a47-1d5b-4d9a-bc3f-0801ea7c72db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:33.745468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-32fd0a47-1d5b-4d9a-bc3f-0801ea7c72db None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:48:33.746669 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-32fd0a47-1d5b-4d9a-bc3f-0801ea7c72db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:48:33.754723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 18:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:48:33.829311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6f6df-dd01-48f4-81fe-68199752a298 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:48:33.830985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a7a6f6df-dd01-48f4-81fe-68199752a298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:48:33.911418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6f6df-dd01-48f4-81fe-68199752a298 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:48:33.915102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc9d28-7c5c-416a-8462-9f5dc99d4727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:48:33.915755 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc9d28-7c5c-416a-8462-9f5dc99d4727 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:33.916099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc9d28-7c5c-416a-8462-9f5dc99d4727 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:33.916376 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc9d28-7c5c-416a-8462-9f5dc99d4727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:33.925636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc9d28-7c5c-416a-8462-9f5dc99d4727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:48:33.927071 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc9d28-7c5c-416a-8462-9f5dc99d4727 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:48:33.936934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 759/1530] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:48:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:48:35.953600 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df21e8b-e3b6-4dd6-986c-80eaa0e0b0d5 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:35.954112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df21e8b-e3b6-4dd6-986c-80eaa0e0b0d5 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:35.954220 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df21e8b-e3b6-4dd6-986c-80eaa0e0b0d5 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:35.955522 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 18:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:48:35.960748 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c2c10-4d29-4c8f-bd17-19b55a25fbdc None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:35.961161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c2c10-4d29-4c8f-bd17-19b55a25fbdc None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:35.961294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c2c10-4d29-4c8f-bd17-19b55a25fbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:35.981065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-1f6c2c10-4d29-4c8f-bd17-19b55a25fbdc None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:48:35.982259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1f6c2c10-4d29-4c8f-bd17-19b55a25fbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:48:35.990113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 18:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:48:36.028758 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df21e8b-e3b6-4dd6-986c-80eaa0e0b0d5 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:48:36.040147 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df21e8b-e3b6-4dd6-986c-80eaa0e0b0d5 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:48:36.044467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036bf36-59fc-4590-ba79-96f1ce477d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:48:36.045081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036bf36-59fc-4590-ba79-96f1ce477d49 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:36.045361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036bf36-59fc-4590-ba79-96f1ce477d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:36.045636 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036bf36-59fc-4590-ba79-96f1ce477d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:36.054796 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036bf36-59fc-4590-ba79-96f1ce477d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:48:36.056199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036bf36-59fc-4590-ba79-96f1ce477d49 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:48:36.065619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 773/1533] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:48:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:48:39.601165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b669ab-6500-4d08-8b2d-8c835173698f None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:39.601741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b669ab-6500-4d08-8b2d-8c835173698f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:39.601741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b669ab-6500-4d08-8b2d-8c835173698f None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:39.603137 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 18:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:48:39.608534 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a7bfc-3f56-4b20-b9f4-584bf8e24767 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:39.608976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a7bfc-3f56-4b20-b9f4-584bf8e24767 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:39.609083 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a7bfc-3f56-4b20-b9f4-584bf8e24767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:39.628881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-673a7bfc-3f56-4b20-b9f4-584bf8e24767 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:48:39.630339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-673a7bfc-3f56-4b20-b9f4-584bf8e24767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:48:39.639124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 18:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:48:39.948690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b669ab-6500-4d08-8b2d-8c835173698f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:48:39.950445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-89b669ab-6500-4d08-8b2d-8c835173698f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:48:40.017024 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:48:40.031069 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b669ab-6500-4d08-8b2d-8c835173698f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:48:40.034626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed760de7-7127-41cb-bd6b-14a4155dbfe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:48:40.035253 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed760de7-7127-41cb-bd6b-14a4155dbfe8 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:40.035536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed760de7-7127-41cb-bd6b-14a4155dbfe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:40.035814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed760de7-7127-41cb-bd6b-14a4155dbfe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:40.045864 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed760de7-7127-41cb-bd6b-14a4155dbfe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:48:40.047299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed760de7-7127-41cb-bd6b-14a4155dbfe8 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:48:40.056997 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 762/1536] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:48:42.107791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c455d76-160c-45f6-88b9-509d000c0f36 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:42.108366 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c455d76-160c-45f6-88b9-509d000c0f36 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:42.108366 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c455d76-160c-45f6-88b9-509d000c0f36 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:42.109738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 18:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:48:42.114940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba103a8-dbf4-4dea-8774-def85b810dd7 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:42.115351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba103a8-dbf4-4dea-8774-def85b810dd7 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:42.115879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba103a8-dbf4-4dea-8774-def85b810dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:42.138200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-0ba103a8-dbf4-4dea-8774-def85b810dd7 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:48:42.139395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-0ba103a8-dbf4-4dea-8774-def85b810dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:48:42.147245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 18:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:48:42.455873 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c455d76-160c-45f6-88b9-509d000c0f36 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:48:42.457597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7c455d76-160c-45f6-88b9-509d000c0f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:48:42.466288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=84 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:48:42.472543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c455d76-160c-45f6-88b9-509d000c0f36 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:48:42.476079 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2504f-2089-4738-94f8-94287d97bec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:48:42.476727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2504f-2089-4738-94f8-94287d97bec1 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:48:42.477015 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2504f-2089-4738-94f8-94287d97bec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:48:42.477288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2504f-2089-4738-94f8-94287d97bec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:48:42.487417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2504f-2089-4738-94f8-94287d97bec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:48:42.488897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2504f-2089-4738-94f8-94287d97bec1 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:48:42.498829 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 776/1539] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:48:52.131354 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=11 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:48:52.494418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=5 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:07.774476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8b7ff-9022-4e53-b445-d0ca14bd0e11 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:07.775804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8b7ff-9022-4e53-b445-d0ca14bd0e11 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:07.776374 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8b7ff-9022-4e53-b445-d0ca14bd0e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:07.806132 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f7a8b7ff-9022-4e53-b445-d0ca14bd0e11 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:07.807470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f7a8b7ff-9022-4e53-b445-d0ca14bd0e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:07.822874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 764/1540] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:07.833623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8569e5f-b2bc-4486-ad8a-7febb541f421 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:07.837006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8569e5f-b2bc-4486-ad8a-7febb541f421 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:07.837006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8569e5f-b2bc-4486-ad8a-7febb541f421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:07.870126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d8569e5f-b2bc-4486-ad8a-7febb541f421 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:07.871356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d8569e5f-b2bc-4486-ad8a-7febb541f421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:07.882804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 777/1541] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:50:07.887678 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8b7ff-9022-4e53-b445-d0ca14bd0e11 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:07.903072 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8b7ff-9022-4e53-b445-d0ca14bd0e11 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:07.915911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7f9c9-1dc1-4a3f-8759-eb83959e924f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:07.916638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7f9c9-1dc1-4a3f-8759-eb83959e924f admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:07.916958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7f9c9-1dc1-4a3f-8759-eb83959e924f admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:07.917240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7f9c9-1dc1-4a3f-8759-eb83959e924f admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:07.918005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7f9c9-1dc1-4a3f-8759-eb83959e924f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:07.918349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7f9c9-1dc1-4a3f-8759-eb83959e924f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:07.920133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7f9c9-1dc1-4a3f-8759-eb83959e924f admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:07.933884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16006228-17b8-4cf5-bc8c-6caf38a74bea None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:07.934217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16006228-17b8-4cf5-bc8c-6caf38a74bea None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:07.934449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16006228-17b8-4cf5-bc8c-6caf38a74bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:07.944261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 765/1542] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:07.954107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7f9c9-1dc1-4a3f-8759-eb83959e924f admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:07.971503 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7f9c9-1dc1-4a3f-8759-eb83959e924f admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:07.972529 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-16006228-17b8-4cf5-bc8c-6caf38a74bea None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:07.974683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-16006228-17b8-4cf5-bc8c-6caf38a74bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:07.981505 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7e539-4eb8-4895-b46e-f35dbf2251c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:07.982214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7e539-4eb8-4895-b46e-f35dbf2251c0 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:07.982478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7e539-4eb8-4895-b46e-f35dbf2251c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:07.982761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7e539-4eb8-4895-b46e-f35dbf2251c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:07.983556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7e539-4eb8-4895-b46e-f35dbf2251c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:07.989509 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: WARNING py.warnings [None req-40b7e539-4eb8-4895-b46e-f35dbf2251c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 18:50:07.989509 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: warnings.warn(msg) Aug 03 18:50:07.989509 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: Aug 03 18:50:07.990807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7e539-4eb8-4895-b46e-f35dbf2251c0 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.007385 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 778/1543] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:08.009318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c3fc8-ddb3-43a6-86f9-653132f8d3d7 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.009876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c3fc8-ddb3-43a6-86f9-653132f8d3d7 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.010368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c3fc8-ddb3-43a6-86f9-653132f8d3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.045424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-8a7c3fc8-ddb3-43a6-86f9-653132f8d3d7 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:08.047690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8a7c3fc8-ddb3-43a6-86f9-653132f8d3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:08.064538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 779/1544] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:08.069261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071f3c9-bc8d-453e-a294-38ddb32ac78a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.069261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071f3c9-bc8d-453e-a294-38ddb32ac78a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.069261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071f3c9-bc8d-453e-a294-38ddb32ac78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.075649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 766/1545] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:07 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:08.077070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba34704-0bb5-48fa-aebd-62618c471a87 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.077366 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba34704-0bb5-48fa-aebd-62618c471a87 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.077686 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba34704-0bb5-48fa-aebd-62618c471a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.110926 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-0071f3c9-bc8d-453e-a294-38ddb32ac78a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:08.112250 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0071f3c9-bc8d-453e-a294-38ddb32ac78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:08.119409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-fba34704-0bb5-48fa-aebd-62618c471a87 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:08.120637 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-fba34704-0bb5-48fa-aebd-62618c471a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:08.204664 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 780/1546] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:08.206063 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1bc0e-abe6-4566-895a-374777fa462e None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.206357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1bc0e-abe6-4566-895a-374777fa462e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.206638 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1bc0e-abe6-4566-895a-374777fa462e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.208227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 767/1547] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:08.209051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba34704-0bb5-48fa-aebd-62618c471a87 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.219629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba34704-0bb5-48fa-aebd-62618c471a87 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.224474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cf1eb-5406-4b90-b0ff-07a84eadc043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.225084 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cf1eb-5406-4b90-b0ff-07a84eadc043 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.225501 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cf1eb-5406-4b90-b0ff-07a84eadc043 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.225784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cf1eb-5406-4b90-b0ff-07a84eadc043 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.226449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cf1eb-5406-4b90-b0ff-07a84eadc043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.228448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cf1eb-5406-4b90-b0ff-07a84eadc043 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.229758 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-13c1bc0e-abe6-4566-895a-374777fa462e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:08.230955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-13c1bc0e-abe6-4566-895a-374777fa462e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:08.239417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 781/1548] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:08.240648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6abc13-a27f-489a-8fa8-f7dda303c705 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.240934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6abc13-a27f-489a-8fa8-f7dda303c705 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.241221 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6abc13-a27f-489a-8fa8-f7dda303c705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.250645 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 768/1549] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:08.253893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c293c-b387-4053-9cd9-80274bf22cc2 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.254050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c293c-b387-4053-9cd9-80274bf22cc2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.254660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c293c-b387-4053-9cd9-80274bf22cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.261914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-7f6abc13-a27f-489a-8fa8-f7dda303c705 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:08.263111 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7f6abc13-a27f-489a-8fa8-f7dda303c705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:08.271818 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 782/1550] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:08.273212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee730b6-2436-4024-8173-90f5299f9975 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.273503 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee730b6-2436-4024-8173-90f5299f9975 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.273776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee730b6-2436-4024-8173-90f5299f9975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.275047 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-cc9c293c-b387-4053-9cd9-80274bf22cc2 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:08.276345 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-cc9c293c-b387-4053-9cd9-80274bf22cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:08.285266 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 769/1551] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:08.286508 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d990581-9e3f-4b26-9240-de577f8c01ac None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.286797 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d990581-9e3f-4b26-9240-de577f8c01ac None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.287075 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d990581-9e3f-4b26-9240-de577f8c01ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.294091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-7ee730b6-2436-4024-8173-90f5299f9975 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:08.296153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7ee730b6-2436-4024-8173-90f5299f9975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:08.303667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 783/1552] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:08.304700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee730b6-2436-4024-8173-90f5299f9975 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.311949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-0d990581-9e3f-4b26-9240-de577f8c01ac None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:08.313767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-0d990581-9e3f-4b26-9240-de577f8c01ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:08.315227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee730b6-2436-4024-8173-90f5299f9975 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.320215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dccc8a-ab33-48d7-bf46-66c410f9fc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.320865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dccc8a-ab33-48d7-bf46-66c410f9fc65 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.321149 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dccc8a-ab33-48d7-bf46-66c410f9fc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.321427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dccc8a-ab33-48d7-bf46-66c410f9fc65 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.321963 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 770/1553] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:08.322136 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dccc8a-ab33-48d7-bf46-66c410f9fc65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:08.322424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dccc8a-ab33-48d7-bf46-66c410f9fc65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.323185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ba91e-07b5-4688-9b18-096608ecfbf2 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.323475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ba91e-07b5-4688-9b18-096608ecfbf2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.323589 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: WARNING py.warnings [None req-e1dccc8a-ab33-48d7-bf46-66c410f9fc65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 18:50:08.323589 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: warnings.warn(msg) Aug 03 18:50:08.323589 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: Aug 03 18:50:08.323792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ba91e-07b5-4688-9b18-096608ecfbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.324418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dccc8a-ab33-48d7-bf46-66c410f9fc65 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.340144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 784/1554] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:08.343885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-852ba91e-07b5-4688-9b18-096608ecfbf2 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:08.344629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dccc8a-ab33-48d7-bf46-66c410f9fc65 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.345113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-852ba91e-07b5-4688-9b18-096608ecfbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:08.353373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 771/1555] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:08.354318 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ba91e-07b5-4688-9b18-096608ecfbf2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.355985 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dccc8a-ab33-48d7-bf46-66c410f9fc65 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.361229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4814d-ccc8-4d16-9e61-02b7dca35225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.361884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4814d-ccc8-4d16-9e61-02b7dca35225 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.362169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4814d-ccc8-4d16-9e61-02b7dca35225 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.362443 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4814d-ccc8-4d16-9e61-02b7dca35225 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.363175 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4814d-ccc8-4d16-9e61-02b7dca35225 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:08.363636 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4814d-ccc8-4d16-9e61-02b7dca35225 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.364179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ba91e-07b5-4688-9b18-096608ecfbf2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.365320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4814d-ccc8-4d16-9e61-02b7dca35225 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.368925 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c4236-b950-4855-ba7b-6a0722d2c735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.369543 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c4236-b950-4855-ba7b-6a0722d2c735 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.369825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c4236-b950-4855-ba7b-6a0722d2c735 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.370347 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c4236-b950-4855-ba7b-6a0722d2c735 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.371061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c4236-b950-4855-ba7b-6a0722d2c735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.372698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c4236-b950-4855-ba7b-6a0722d2c735 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.378648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 785/1556] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:08.380377 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088aa316-0029-4045-af37-c4141a12f774 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.380668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088aa316-0029-4045-af37-c4141a12f774 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.380945 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088aa316-0029-4045-af37-c4141a12f774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.388770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 772/1557] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:08.388770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469e141-d713-46e6-9cd4-95537b97e8e5 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.388770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469e141-d713-46e6-9cd4-95537b97e8e5 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.388770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469e141-d713-46e6-9cd4-95537b97e8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.404171 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-088aa316-0029-4045-af37-c4141a12f774 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:08.405413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-088aa316-0029-4045-af37-c4141a12f774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:08.407116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f469e141-d713-46e6-9cd4-95537b97e8e5 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:08.408300 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f469e141-d713-46e6-9cd4-95537b97e8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:08.413481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 786/1558] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:08.414664 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eba270-2a99-450d-9cdb-48dfe47f81ce None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.414949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eba270-2a99-450d-9cdb-48dfe47f81ce None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.415225 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eba270-2a99-450d-9cdb-48dfe47f81ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.416387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 773/1559] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:08.417666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2252a0-89dc-4a25-be6e-65fe184166c0 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.417958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2252a0-89dc-4a25-be6e-65fe184166c0 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.418234 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2252a0-89dc-4a25-be6e-65fe184166c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.438019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-bd2252a0-89dc-4a25-be6e-65fe184166c0 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:08.439218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-bd2252a0-89dc-4a25-be6e-65fe184166c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:08.439691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-a3eba270-2a99-450d-9cdb-48dfe47f81ce None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:08.440872 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a3eba270-2a99-450d-9cdb-48dfe47f81ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:08.447588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 774/1560] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:08.448435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2252a0-89dc-4a25-be6e-65fe184166c0 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.448989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 787/1561] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:08.449571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eba270-2a99-450d-9cdb-48dfe47f81ce None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.459953 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eba270-2a99-450d-9cdb-48dfe47f81ce None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.462014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2252a0-89dc-4a25-be6e-65fe184166c0 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.464650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fddef-10f7-4219-b587-ea644bfab3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.465256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fddef-10f7-4219-b587-ea644bfab3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.465534 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fddef-10f7-4219-b587-ea644bfab3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.465806 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fddef-10f7-4219-b587-ea644bfab3e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.466502 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fddef-10f7-4219-b587-ea644bfab3e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:08.466657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d07b3-c7ef-431f-a218-3fd3d59efc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.466789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fddef-10f7-4219-b587-ea644bfab3e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.467280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d07b3-c7ef-431f-a218-3fd3d59efc40 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.467546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d07b3-c7ef-431f-a218-3fd3d59efc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.467822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d07b3-c7ef-431f-a218-3fd3d59efc40 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.468669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d07b3-c7ef-431f-a218-3fd3d59efc40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.468845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fddef-10f7-4219-b587-ea644bfab3e0 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.471155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d07b3-c7ef-431f-a218-3fd3d59efc40 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.480903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 788/1562] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:08.481709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fddef-10f7-4219-b587-ea644bfab3e0 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.494955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fddef-10f7-4219-b587-ea644bfab3e0 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.500313 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264225f-e9e3-423c-8b4c-167489bca0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.503841 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 775/1563] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:08.504023 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264225f-e9e3-423c-8b4c-167489bca0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.504023 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264225f-e9e3-423c-8b4c-167489bca0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.504503 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d07b3-c7ef-431f-a218-3fd3d59efc40 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.504612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264225f-e9e3-423c-8b4c-167489bca0b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.504612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264225f-e9e3-423c-8b4c-167489bca0b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.504612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264225f-e9e3-423c-8b4c-167489bca0b4 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.532681 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:08.533479 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 789/1564] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:08.534516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264225f-e9e3-423c-8b4c-167489bca0b4 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.572180 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d07b3-c7ef-431f-a218-3fd3d59efc40 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.591736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:08.607237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264225f-e9e3-423c-8b4c-167489bca0b4 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.612164 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7612a-78ce-4c17-abdc-9023960fc5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.612816 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7612a-78ce-4c17-abdc-9023960fc5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.613100 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7612a-78ce-4c17-abdc-9023960fc5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.613522 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7612a-78ce-4c17-abdc-9023960fc5c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.614281 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f7612a-78ce-4c17-abdc-9023960fc5c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:08.614582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f7612a-78ce-4c17-abdc-9023960fc5c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.616192 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f7612a-78ce-4c17-abdc-9023960fc5c1 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.628743 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 790/1565] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:08.630073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7612a-78ce-4c17-abdc-9023960fc5c1 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.633644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cb549-fdd9-4486-8ab5-145087840801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.634281 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cb549-fdd9-4486-8ab5-145087840801 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.634567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cb549-fdd9-4486-8ab5-145087840801 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.634847 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cb549-fdd9-4486-8ab5-145087840801 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.635556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56cb549-fdd9-4486-8ab5-145087840801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.637344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56cb549-fdd9-4486-8ab5-145087840801 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.648013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7612a-78ce-4c17-abdc-9023960fc5c1 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.652230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 776/1566] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:08.652230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cb549-fdd9-4486-8ab5-145087840801 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.652667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea088d-e63f-49a4-b530-96f38203f0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.653277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea088d-e63f-49a4-b530-96f38203f0be admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.653559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea088d-e63f-49a4-b530-96f38203f0be admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.653859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea088d-e63f-49a4-b530-96f38203f0be admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.654555 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea088d-e63f-49a4-b530-96f38203f0be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:08.654974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea088d-e63f-49a4-b530-96f38203f0be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.657129 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea088d-e63f-49a4-b530-96f38203f0be admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.660017 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cb549-fdd9-4486-8ab5-145087840801 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.664702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3146358-7d20-4aea-9dbc-075de4bb5350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.665320 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3146358-7d20-4aea-9dbc-075de4bb5350 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.665604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3146358-7d20-4aea-9dbc-075de4bb5350 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.665884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3146358-7d20-4aea-9dbc-075de4bb5350 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.666591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3146358-7d20-4aea-9dbc-075de4bb5350 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:08.666892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3146358-7d20-4aea-9dbc-075de4bb5350 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.668501 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3146358-7d20-4aea-9dbc-075de4bb5350 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.669805 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 791/1567] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:08.670630 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea088d-e63f-49a4-b530-96f38203f0be admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.687173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea088d-e63f-49a4-b530-96f38203f0be admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.689845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 777/1568] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:08.690602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3146358-7d20-4aea-9dbc-075de4bb5350 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.695685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423797de-7870-452f-953e-421449d172ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.696319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423797de-7870-452f-953e-421449d172ff admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.696600 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423797de-7870-452f-953e-421449d172ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.696868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423797de-7870-452f-953e-421449d172ff admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.697595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423797de-7870-452f-953e-421449d172ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:08.697900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423797de-7870-452f-953e-421449d172ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.699498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423797de-7870-452f-953e-421449d172ff admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.701881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3146358-7d20-4aea-9dbc-075de4bb5350 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.707270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4613d6f-bb4c-4716-9c78-482979e6f0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.707890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4613d6f-bb4c-4716-9c78-482979e6f0f9 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.708173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4613d6f-bb4c-4716-9c78-482979e6f0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.708672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4613d6f-bb4c-4716-9c78-482979e6f0f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.709342 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4613d6f-bb4c-4716-9c78-482979e6f0f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.711233 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4613d6f-bb4c-4716-9c78-482979e6f0f9 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.716784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 792/1569] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:08.716784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423797de-7870-452f-953e-421449d172ff admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.729398 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423797de-7870-452f-953e-421449d172ff admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.735194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca0776-3f43-4914-b100-2fcd6122f53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.736109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca0776-3f43-4914-b100-2fcd6122f53f admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.736422 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca0776-3f43-4914-b100-2fcd6122f53f admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.736701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca0776-3f43-4914-b100-2fcd6122f53f admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.737377 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca0776-3f43-4914-b100-2fcd6122f53f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.738402 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: WARNING py.warnings [None req-d7ca0776-3f43-4914-b100-2fcd6122f53f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 18:50:08.738402 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: warnings.warn(msg) Aug 03 18:50:08.738402 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: Aug 03 18:50:08.739616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca0776-3f43-4914-b100-2fcd6122f53f admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.743212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 778/1570] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:08.743212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4613d6f-bb4c-4716-9c78-482979e6f0f9 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.763012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 793/1571] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/users => generated 284 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:08.763840 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca0776-3f43-4914-b100-2fcd6122f53f admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.773793 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca0776-3f43-4914-b100-2fcd6122f53f admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.811045 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4613d6f-bb4c-4716-9c78-482979e6f0f9 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.816756 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c85033-b805-4b23-bb79-3100295b9590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.817383 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c85033-b805-4b23-bb79-3100295b9590 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.817668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c85033-b805-4b23-bb79-3100295b9590 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.818008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c85033-b805-4b23-bb79-3100295b9590 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.818952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c85033-b805-4b23-bb79-3100295b9590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.820921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c85033-b805-4b23-bb79-3100295b9590 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.833754 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f788cf-5951-4ac5-bb91-1437aaabe6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.834344 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f788cf-5951-4ac5-bb91-1437aaabe6c8 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.834632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f788cf-5951-4ac5-bb91-1437aaabe6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.835099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f788cf-5951-4ac5-bb91-1437aaabe6c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.835886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f788cf-5951-4ac5-bb91-1437aaabe6c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.837904 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f788cf-5951-4ac5-bb91-1437aaabe6c8 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.842908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 779/1572] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:08.843869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c85033-b805-4b23-bb79-3100295b9590 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.854210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c85033-b805-4b23-bb79-3100295b9590 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.859875 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028ac10-6e17-4809-b976-1e8a150cfa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.860655 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028ac10-6e17-4809-b976-1e8a150cfa70 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.860941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028ac10-6e17-4809-b976-1e8a150cfa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.861282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028ac10-6e17-4809-b976-1e8a150cfa70 admin admin] PATH_INFO: `/v3/projects/a9ad699f792e4c50a898079291f09254/users/1f99fcc5eddd45c89abca70f00ad6fed/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.871340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 794/1573] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:08.875215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f788cf-5951-4ac5-bb91-1437aaabe6c8 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.878853 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028ac10-6e17-4809-b976-1e8a150cfa70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9ad699f792e4c50a898079291f09254, user_id=1f99fcc5eddd45c89abca70f00ad6fed, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.880174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: WARNING py.warnings [None req-3028ac10-6e17-4809-b976-1e8a150cfa70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 18:50:08.880174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: warnings.warn(msg) Aug 03 18:50:08.880174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: Aug 03 18:50:08.882171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028ac10-6e17-4809-b976-1e8a150cfa70 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:08.891325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:08.896318 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 780/1574] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:08 2022] PUT /identity/v3/projects/a9ad699f792e4c50a898079291f09254/users/1f99fcc5eddd45c89abca70f00ad6fed/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:08.897274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028ac10-6e17-4809-b976-1e8a150cfa70 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:08.944199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f788cf-5951-4ac5-bb91-1437aaabe6c8 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.965357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028ac10-6e17-4809-b976-1e8a150cfa70 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:08.971051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480c5a1-4b6d-4318-add3-1d9b42ed1cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:08.971668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:08.971813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480c5a1-4b6d-4318-add3-1d9b42ed1cfd admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:08.971955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480c5a1-4b6d-4318-add3-1d9b42ed1cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:08.972235 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480c5a1-4b6d-4318-add3-1d9b42ed1cfd admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:08.973323 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480c5a1-4b6d-4318-add3-1d9b42ed1cfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:08.975392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480c5a1-4b6d-4318-add3-1d9b42ed1cfd admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.006984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 781/1575] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:09.006984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480c5a1-4b6d-4318-add3-1d9b42ed1cfd admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.008035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473f48d-393a-40cb-8200-d2e68926df39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.008700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473f48d-393a-40cb-8200-d2e68926df39 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.008981 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473f48d-393a-40cb-8200-d2e68926df39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.009253 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473f48d-393a-40cb-8200-d2e68926df39 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.009919 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473f48d-393a-40cb-8200-d2e68926df39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.011840 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473f48d-393a-40cb-8200-d2e68926df39 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.038522 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 795/1576] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:08 2022] POST /identity/v3/projects => generated 405 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:09.039626 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473f48d-393a-40cb-8200-d2e68926df39 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.076740 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480c5a1-4b6d-4318-add3-1d9b42ed1cfd admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.104651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473f48d-393a-40cb-8200-d2e68926df39 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.109882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a72337-2081-46f3-ae0a-6c29e2f902a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.110456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a72337-2081-46f3-ae0a-6c29e2f902a5 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.110735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a72337-2081-46f3-ae0a-6c29e2f902a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.111012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a72337-2081-46f3-ae0a-6c29e2f902a5 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.111674 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a72337-2081-46f3-ae0a-6c29e2f902a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.113674 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a72337-2081-46f3-ae0a-6c29e2f902a5 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.134573 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd3c28-7721-4008-acd6-8681be5d35c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.135226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd3c28-7721-4008-acd6-8681be5d35c7 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.135534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd3c28-7721-4008-acd6-8681be5d35c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.135814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd3c28-7721-4008-acd6-8681be5d35c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.139511 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 796/1577] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:09 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:09.139665 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd3c28-7721-4008-acd6-8681be5d35c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.139764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a72337-2081-46f3-ae0a-6c29e2f902a5 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.139864 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd3c28-7721-4008-acd6-8681be5d35c7 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.147187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a72337-2081-46f3-ae0a-6c29e2f902a5 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.153439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c5e80-311d-4828-8313-0dbf08d3d73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.154053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c5e80-311d-4828-8313-0dbf08d3d73f admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.154337 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c5e80-311d-4828-8313-0dbf08d3d73f admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.154619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c5e80-311d-4828-8313-0dbf08d3d73f admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.155324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c5e80-311d-4828-8313-0dbf08d3d73f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.157794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c5e80-311d-4828-8313-0dbf08d3d73f admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.169825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 782/1578] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:09 2022] POST /identity/v3/projects => generated 375 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:09.170898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd3c28-7721-4008-acd6-8681be5d35c7 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.173839 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 797/1579] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:09.174682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c5e80-311d-4828-8313-0dbf08d3d73f admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.188446 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c5e80-311d-4828-8313-0dbf08d3d73f admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.200146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:09.249407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd3c28-7721-4008-acd6-8681be5d35c7 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.254557 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e18e0e-b910-431b-9106-bd6b56e65814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.255045 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03442a71-998a-41be-95c1-e40033e22ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.255181 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e18e0e-b910-431b-9106-bd6b56e65814 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.255443 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e18e0e-b910-431b-9106-bd6b56e65814 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.255711 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e18e0e-b910-431b-9106-bd6b56e65814 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.255815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03442a71-998a-41be-95c1-e40033e22ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.255998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03442a71-998a-41be-95c1-e40033e22ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.256277 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03442a71-998a-41be-95c1-e40033e22ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.256413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e18e0e-b910-431b-9106-bd6b56e65814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.257043 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03442a71-998a-41be-95c1-e40033e22ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.258363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e18e0e-b910-431b-9106-bd6b56e65814 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.258675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03442a71-998a-41be-95c1-e40033e22ab3 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.270190 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 783/1580] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:09.271454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce9e5c-8c68-4182-ae7c-e43d0669d485 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.271799 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce9e5c-8c68-4182-ae7c-e43d0669d485 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.272119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce9e5c-8c68-4182-ae7c-e43d0669d485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.283369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 798/1581] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:09 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:09.284197 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e18e0e-b910-431b-9106-bd6b56e65814 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.293373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f6ce9e5c-8c68-4182-ae7c-e43d0669d485 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:09.293621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e18e0e-b910-431b-9106-bd6b56e65814 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.294583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f6ce9e5c-8c68-4182-ae7c-e43d0669d485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:09.298505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd706736-8d32-4182-8d56-af7d1ad7f479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.299117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd706736-8d32-4182-8d56-af7d1ad7f479 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.299401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd706736-8d32-4182-8d56-af7d1ad7f479 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.299674 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd706736-8d32-4182-8d56-af7d1ad7f479 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.300404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd706736-8d32-4182-8d56-af7d1ad7f479 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.302366 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd706736-8d32-4182-8d56-af7d1ad7f479 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.302708 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 784/1582] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:09.303474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce9e5c-8c68-4182-ae7c-e43d0669d485 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.314601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce9e5c-8c68-4182-ae7c-e43d0669d485 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.324634 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ff753-263b-4ea3-ac3b-3828c4dbaed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.325366 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ff753-263b-4ea3-ac3b-3828c4dbaed7 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.325720 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ff753-263b-4ea3-ac3b-3828c4dbaed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.326068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ff753-263b-4ea3-ac3b-3828c4dbaed7 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.326760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ff753-263b-4ea3-ac3b-3828c4dbaed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.330360 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ff753-263b-4ea3-ac3b-3828c4dbaed7 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.333766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 799/1583] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:09 2022] POST /identity/v3/users => generated 290 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:09.333766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd706736-8d32-4182-8d56-af7d1ad7f479 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.344049 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:09.348839 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd706736-8d32-4182-8d56-af7d1ad7f479 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.356814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 785/1584] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:09 2022] POST /identity/v3/users => generated 303 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:09.356814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ff753-263b-4ea3-ac3b-3828c4dbaed7 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.357066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81753400-5a52-4d3a-a880-80a68b1534a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.357066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81753400-5a52-4d3a-a880-80a68b1534a7 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.357066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81753400-5a52-4d3a-a880-80a68b1534a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.357066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81753400-5a52-4d3a-a880-80a68b1534a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.357382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81753400-5a52-4d3a-a880-80a68b1534a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.358583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81753400-5a52-4d3a-a880-80a68b1534a7 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.364743 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ff753-263b-4ea3-ac3b-3828c4dbaed7 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.369252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf13a45-4884-4921-9875-dcaaf4653e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.369859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf13a45-4884-4921-9875-dcaaf4653e72 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.370167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf13a45-4884-4921-9875-dcaaf4653e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.370442 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf13a45-4884-4921-9875-dcaaf4653e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.371134 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf13a45-4884-4921-9875-dcaaf4653e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.371384 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 800/1585] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:09.372172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81753400-5a52-4d3a-a880-80a68b1534a7 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.372767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf13a45-4884-4921-9875-dcaaf4653e72 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.384807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 786/1586] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:09.388203 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81753400-5a52-4d3a-a880-80a68b1534a7 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.388337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf13a45-4884-4921-9875-dcaaf4653e72 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.390865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa60a2-f9be-4564-a984-352131be2716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.391700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa60a2-f9be-4564-a984-352131be2716 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.391984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa60a2-f9be-4564-a984-352131be2716 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.392258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa60a2-f9be-4564-a984-352131be2716 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.393617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa60a2-f9be-4564-a984-352131be2716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.395588 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa60a2-f9be-4564-a984-352131be2716 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.396506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf13a45-4884-4921-9875-dcaaf4653e72 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.401005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc789c-7076-47ed-8a59-d024875dd6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.401601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc789c-7076-47ed-8a59-d024875dd6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.401880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc789c-7076-47ed-8a59-d024875dd6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.402151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc789c-7076-47ed-8a59-d024875dd6c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.402833 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cc789c-7076-47ed-8a59-d024875dd6c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.404440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cc789c-7076-47ed-8a59-d024875dd6c0 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.416068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 787/1587] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:09.416770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc789c-7076-47ed-8a59-d024875dd6c0 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.418607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 801/1588] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:09 2022] POST /identity/v3/users => generated 288 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:09.419435 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa60a2-f9be-4564-a984-352131be2716 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.431591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc789c-7076-47ed-8a59-d024875dd6c0 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.436310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6524cd7-d229-4ca0-8f24-7e002618d47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.436961 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6524cd7-d229-4ca0-8f24-7e002618d47b admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.438012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6524cd7-d229-4ca0-8f24-7e002618d47b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.438140 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6524cd7-d229-4ca0-8f24-7e002618d47b admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.438741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6524cd7-d229-4ca0-8f24-7e002618d47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.440366 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6524cd7-d229-4ca0-8f24-7e002618d47b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.454264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 788/1589] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:09.454264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6524cd7-d229-4ca0-8f24-7e002618d47b admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.462243 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6524cd7-d229-4ca0-8f24-7e002618d47b admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.467253 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6eb81-b79a-4bc0-b015-e8179e1f033b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.467856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6eb81-b79a-4bc0-b015-e8179e1f033b admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.468141 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6eb81-b79a-4bc0-b015-e8179e1f033b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.468651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6eb81-b79a-4bc0-b015-e8179e1f033b admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.469351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6eb81-b79a-4bc0-b015-e8179e1f033b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.470949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6eb81-b79a-4bc0-b015-e8179e1f033b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.477653 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa60a2-f9be-4564-a984-352131be2716 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.481003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 789/1590] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:09.481535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65ca05-ce1c-4ad5-94c3-d64eaeee9b44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.481796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6eb81-b79a-4bc0-b015-e8179e1f033b admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.482536 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65ca05-ce1c-4ad5-94c3-d64eaeee9b44 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.483047 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65ca05-ce1c-4ad5-94c3-d64eaeee9b44 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.483448 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65ca05-ce1c-4ad5-94c3-d64eaeee9b44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.491804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6eb81-b79a-4bc0-b015-e8179e1f033b admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.494058 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65ca05-ce1c-4ad5-94c3-d64eaeee9b44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.495494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65ca05-ce1c-4ad5-94c3-d64eaeee9b44 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.496416 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2063f-4660-48fd-a918-16ce76e5e54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.497027 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2063f-4660-48fd-a918-16ce76e5e54a admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.497304 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2063f-4660-48fd-a918-16ce76e5e54a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.497580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2063f-4660-48fd-a918-16ce76e5e54a admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.498262 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2063f-4660-48fd-a918-16ce76e5e54a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.500729 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2063f-4660-48fd-a918-16ce76e5e54a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.506280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 802/1591] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:09.513739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 790/1592] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:09.514039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65ca05-ce1c-4ad5-94c3-d64eaeee9b44 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.514156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2063f-4660-48fd-a918-16ce76e5e54a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.523681 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65ca05-ce1c-4ad5-94c3-d64eaeee9b44 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.525364 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2063f-4660-48fd-a918-16ce76e5e54a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.528515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90bd5e-0785-4c62-aeb9-11da29a67100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.529241 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90bd5e-0785-4c62-aeb9-11da29a67100 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.529506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90bd5e-0785-4c62-aeb9-11da29a67100 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.529749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90bd5e-0785-4c62-aeb9-11da29a67100 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.530428 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51570017-60d9-48ef-80b9-995494302031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.530573 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c90bd5e-0785-4c62-aeb9-11da29a67100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.531227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51570017-60d9-48ef-80b9-995494302031 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.531576 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51570017-60d9-48ef-80b9-995494302031 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.531940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51570017-60d9-48ef-80b9-995494302031 admin admin] PATH_INFO: `/v3/projects/b16c594fd667419ba6d04396b3c730ef/users/e01aa12142eb49d78094cbc5c740d62c/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.532339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c90bd5e-0785-4c62-aeb9-11da29a67100 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.542879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 803/1593] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:09.543690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90bd5e-0785-4c62-aeb9-11da29a67100 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.552344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51570017-60d9-48ef-80b9-995494302031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16c594fd667419ba6d04396b3c730ef, user_id=e01aa12142eb49d78094cbc5c740d62c, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.555172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51570017-60d9-48ef-80b9-995494302031 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.558459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90bd5e-0785-4c62-aeb9-11da29a67100 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.564763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb2b7c-c194-440f-a093-c435080a9283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.565501 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb2b7c-c194-440f-a093-c435080a9283 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.565827 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb2b7c-c194-440f-a093-c435080a9283 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.566111 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb2b7c-c194-440f-a093-c435080a9283 admin admin] PATH_INFO: `/v3/projects/77d5ff41add94950b7422c6de3e36257/users/13ab977f852a41e898af1ddfb67bad69/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.575956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 791/1594] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:09 2022] PUT /identity/v3/projects/b16c594fd667419ba6d04396b3c730ef/users/e01aa12142eb49d78094cbc5c740d62c/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:09.577102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51570017-60d9-48ef-80b9-995494302031 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.596280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb2b7c-c194-440f-a093-c435080a9283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d5ff41add94950b7422c6de3e36257, user_id=13ab977f852a41e898af1ddfb67bad69, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.601839 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb2b7c-c194-440f-a093-c435080a9283 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.619191 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 804/1595] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:09 2022] PUT /identity/v3/projects/77d5ff41add94950b7422c6de3e36257/users/13ab977f852a41e898af1ddfb67bad69/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:09.620699 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb2b7c-c194-440f-a093-c435080a9283 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.678551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51570017-60d9-48ef-80b9-995494302031 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.693081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:09.712017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb2b7c-c194-440f-a093-c435080a9283 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.721040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3404eff-0af9-49d8-afc6-8532720d5d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.722286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3404eff-0af9-49d8-afc6-8532720d5d62 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.722876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3404eff-0af9-49d8-afc6-8532720d5d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.723375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3404eff-0af9-49d8-afc6-8532720d5d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.724791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3404eff-0af9-49d8-afc6-8532720d5d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.729003 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3404eff-0af9-49d8-afc6-8532720d5d62 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.760338 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 805/1596] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:09.760468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3404eff-0af9-49d8-afc6-8532720d5d62 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.775303 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181f7a9-5df4-4e71-a084-3c093c8f9d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.776105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181f7a9-5df4-4e71-a084-3c093c8f9d35 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.776428 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181f7a9-5df4-4e71-a084-3c093c8f9d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.776719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181f7a9-5df4-4e71-a084-3c093c8f9d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.777427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181f7a9-5df4-4e71-a084-3c093c8f9d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.779053 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181f7a9-5df4-4e71-a084-3c093c8f9d35 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.779243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3404eff-0af9-49d8-afc6-8532720d5d62 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.784910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9251ce-db28-41f2-b6b1-66506ecfe250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.785541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9251ce-db28-41f2-b6b1-66506ecfe250 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.785826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9251ce-db28-41f2-b6b1-66506ecfe250 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.786103 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9251ce-db28-41f2-b6b1-66506ecfe250 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.786803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9251ce-db28-41f2-b6b1-66506ecfe250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.788441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9251ce-db28-41f2-b6b1-66506ecfe250 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.789902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 792/1597] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:09.790891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181f7a9-5df4-4e71-a084-3c093c8f9d35 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.803275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181f7a9-5df4-4e71-a084-3c093c8f9d35 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.807043 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 806/1598] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:09.807043 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9251ce-db28-41f2-b6b1-66506ecfe250 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.809908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5eb37-02bd-4dea-9ce8-ba83a6f2bcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.810196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5eb37-02bd-4dea-9ce8-ba83a6f2bcb7 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.810466 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5eb37-02bd-4dea-9ce8-ba83a6f2bcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.810949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5eb37-02bd-4dea-9ce8-ba83a6f2bcb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.811688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5eb37-02bd-4dea-9ce8-ba83a6f2bcb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.814669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5eb37-02bd-4dea-9ce8-ba83a6f2bcb7 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.817025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9251ce-db28-41f2-b6b1-66506ecfe250 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.822002 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5563c23-4add-40dd-86e0-dc6cded72124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.822670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5563c23-4add-40dd-86e0-dc6cded72124 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.823106 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5563c23-4add-40dd-86e0-dc6cded72124 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.823391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5563c23-4add-40dd-86e0-dc6cded72124 admin admin] PATH_INFO: `/v3/projects/81ddb9c7b0f442df80831cc49287fd57/users/c6085ba9dff1480a8a1674e492e0c6c3/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.828877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 793/1599] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:09.828877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304eaa82-5e21-45c7-b2b2-21cad5fd76f6 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.828877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304eaa82-5e21-45c7-b2b2-21cad5fd76f6 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.828877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304eaa82-5e21-45c7-b2b2-21cad5fd76f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.848544 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5563c23-4add-40dd-86e0-dc6cded72124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ddb9c7b0f442df80831cc49287fd57, user_id=c6085ba9dff1480a8a1674e492e0c6c3, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.851552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5563c23-4add-40dd-86e0-dc6cded72124 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:09.863758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-304eaa82-5e21-45c7-b2b2-21cad5fd76f6 None None] MFA Rules not processed for user `e01aa12142eb49d78094cbc5c740d62c`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:09.865430 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-304eaa82-5e21-45c7-b2b2-21cad5fd76f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:09.869487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 807/1600] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:09 2022] PUT /identity/v3/projects/81ddb9c7b0f442df80831cc49287fd57/users/c6085ba9dff1480a8a1674e492e0c6c3/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:09.874476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672044c-c219-40c9-9943-2bbcd5134a7a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.874764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672044c-c219-40c9-9943-2bbcd5134a7a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.875060 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672044c-c219-40c9-9943-2bbcd5134a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.883578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:09.897405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-1672044c-c219-40c9-9943-2bbcd5134a7a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:09.898644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1672044c-c219-40c9-9943-2bbcd5134a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:09.968813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 808/1601] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:09.970134 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672044c-c219-40c9-9943-2bbcd5134a7a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.970721 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 794/1602] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:09.971949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304eaa82-5e21-45c7-b2b2-21cad5fd76f6 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:09.985628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672044c-c219-40c9-9943-2bbcd5134a7a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.985972 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304eaa82-5e21-45c7-b2b2-21cad5fd76f6 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:09.991745 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8745d06-8c75-464c-b465-762955ede3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.992151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fe6ff-8b44-4502-8b92-9c2d3cc7f178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:09.992494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8745d06-8c75-464c-b465-762955ede3cd admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.992781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8745d06-8c75-464c-b465-762955ede3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.993061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8745d06-8c75-464c-b465-762955ede3cd admin admin] PATH_INFO: `/v3/projects/e125626c9566484b837b582ba7096598/users/1aa1d56d9b154e32aa1c136b577d49f4/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.993518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fe6ff-8b44-4502-8b92-9c2d3cc7f178 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:09.993959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fe6ff-8b44-4502-8b92-9c2d3cc7f178 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:09.994458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fe6ff-8b44-4502-8b92-9c2d3cc7f178 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:09.995374 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fe6ff-8b44-4502-8b92-9c2d3cc7f178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:09.998845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fe6ff-8b44-4502-8b92-9c2d3cc7f178 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:10.012317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8745d06-8c75-464c-b465-762955ede3cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e125626c9566484b837b582ba7096598, user_id=1aa1d56d9b154e32aa1c136b577d49f4, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:10.015080 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8745d06-8c75-464c-b465-762955ede3cd admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:10.016496 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 795/1603] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:10.017625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fe6ff-8b44-4502-8b92-9c2d3cc7f178 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:10.028840 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fe6ff-8b44-4502-8b92-9c2d3cc7f178 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:10.031331 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 809/1604] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:09 2022] PUT /identity/v3/projects/e125626c9566484b837b582ba7096598/users/1aa1d56d9b154e32aa1c136b577d49f4/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:10.032122 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8745d06-8c75-464c-b465-762955ede3cd admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:10.103273 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8745d06-8c75-464c-b465-762955ede3cd admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:10.107289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee11a9d-65a5-4a0e-aa0a-c3bb931bbe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:10.108163 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee11a9d-65a5-4a0e-aa0a-c3bb931bbe4a admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:10.108565 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee11a9d-65a5-4a0e-aa0a-c3bb931bbe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:10.109074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee11a9d-65a5-4a0e-aa0a-c3bb931bbe4a admin admin] PATH_INFO: `/v3/projects/f7d2f4260b1b44c7b145bd0e013f1fda/users/beb3c5d849a743ce969fb5e570c9b94f/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:10.111203 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b1a6c-aabe-46b0-8aa7-954f0a398a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:10.112219 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b1a6c-aabe-46b0-8aa7-954f0a398a02 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:10.112546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b1a6c-aabe-46b0-8aa7-954f0a398a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:10.112828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b1a6c-aabe-46b0-8aa7-954f0a398a02 admin admin] PATH_INFO: `/v3/projects/519094d345e8478bb11fd546fa97bdb2/users/4645272c6f3849ffabd98634a927a461/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:10.126690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee11a9d-65a5-4a0e-aa0a-c3bb931bbe4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d2f4260b1b44c7b145bd0e013f1fda, user_id=beb3c5d849a743ce969fb5e570c9b94f, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:10.130091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee11a9d-65a5-4a0e-aa0a-c3bb931bbe4a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:10.130208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b1a6c-aabe-46b0-8aa7-954f0a398a02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=519094d345e8478bb11fd546fa97bdb2, user_id=4645272c6f3849ffabd98634a927a461, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:10.133013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b1a6c-aabe-46b0-8aa7-954f0a398a02 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:10.148190 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 796/1605] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:10 2022] PUT /identity/v3/projects/f7d2f4260b1b44c7b145bd0e013f1fda/users/beb3c5d849a743ce969fb5e570c9b94f/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:10.149941 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 810/1606] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:10 2022] PUT /identity/v3/projects/519094d345e8478bb11fd546fa97bdb2/users/4645272c6f3849ffabd98634a927a461/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:10.150718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b1a6c-aabe-46b0-8aa7-954f0a398a02 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:10.153690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35227420-84f7-4305-911d-ed62035e761e None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:10.153973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35227420-84f7-4305-911d-ed62035e761e None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:10.154389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35227420-84f7-4305-911d-ed62035e761e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:10.164499 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:10.179886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-35227420-84f7-4305-911d-ed62035e761e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:10.181188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-35227420-84f7-4305-911d-ed62035e761e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:10.231605 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b1a6c-aabe-46b0-8aa7-954f0a398a02 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:10.236095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09d4a9-966a-4b9d-8a2e-f6279dbc02ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:10.236800 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09d4a9-966a-4b9d-8a2e-f6279dbc02ca service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:10.237083 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09d4a9-966a-4b9d-8a2e-f6279dbc02ca service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:10.237604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09d4a9-966a-4b9d-8a2e-f6279dbc02ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:10.250480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 797/1607] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:10.251797 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35227420-84f7-4305-911d-ed62035e761e None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:10.260732 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:10.316765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35227420-84f7-4305-911d-ed62035e761e None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:10.320434 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a41b0-b340-44a4-bbe7-18565796716c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:10.321085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a41b0-b340-44a4-bbe7-18565796716c service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:10.321367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a41b0-b340-44a4-bbe7-18565796716c service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:10.321644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a41b0-b340-44a4-bbe7-18565796716c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:10.325806 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09d4a9-966a-4b9d-8a2e-f6279dbc02ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:10.327358 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09d4a9-966a-4b9d-8a2e-f6279dbc02ca service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:10.339647 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 811/1608] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:10.339647 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09d4a9-966a-4b9d-8a2e-f6279dbc02ca service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:10.349056 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09d4a9-966a-4b9d-8a2e-f6279dbc02ca service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:10.353721 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430fb27-6247-4751-b009-1e95bc5d252f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:10.354357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430fb27-6247-4751-b009-1e95bc5d252f admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:10.354641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430fb27-6247-4751-b009-1e95bc5d252f admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:10.354915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430fb27-6247-4751-b009-1e95bc5d252f admin admin] PATH_INFO: `/v3/projects/8fa50cb14ff74c8c9ec93af52f98ae2b/users/2080b428e34c491d92e7df054c98105b/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:10.375201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430fb27-6247-4751-b009-1e95bc5d252f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa50cb14ff74c8c9ec93af52f98ae2b, user_id=2080b428e34c491d92e7df054c98105b, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:10.378455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430fb27-6247-4751-b009-1e95bc5d252f admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:10.406465 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 812/1609] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:10 2022] PUT /identity/v3/projects/8fa50cb14ff74c8c9ec93af52f98ae2b/users/2080b428e34c491d92e7df054c98105b/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:10.416563 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92e381-dc7f-426b-ab8d-257864187537 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:10.416563 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92e381-dc7f-426b-ab8d-257864187537 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:10.416563 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92e381-dc7f-426b-ab8d-257864187537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:10.422266 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a41b0-b340-44a4-bbe7-18565796716c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:10.424664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a41b0-b340-44a4-bbe7-18565796716c service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:10.432127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-8c92e381-dc7f-426b-ab8d-257864187537 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:10.433533 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8c92e381-dc7f-426b-ab8d-257864187537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:10.445556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 798/1610] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:10.446954 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d99f0-d7b2-45a5-b04f-3e2ed26d4822 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:10.447245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d99f0-d7b2-45a5-b04f-3e2ed26d4822 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:10.447529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d99f0-d7b2-45a5-b04f-3e2ed26d4822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:10.502049 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f92d99f0-d7b2-45a5-b04f-3e2ed26d4822 None None] MFA Rules not processed for user `beb3c5d849a743ce969fb5e570c9b94f`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:10.503491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f92d99f0-d7b2-45a5-b04f-3e2ed26d4822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:10.566770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:10.590260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 813/1611] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:10.590260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bc946-4d70-4e10-9c33-d409d1b64b72 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:10.590260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bc946-4d70-4e10-9c33-d409d1b64b72 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:10.590260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bc946-4d70-4e10-9c33-d409d1b64b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:10.602680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 799/1612] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:10.603621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d99f0-d7b2-45a5-b04f-3e2ed26d4822 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:10.658987 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-a54bc946-4d70-4e10-9c33-d409d1b64b72 None None] MFA Rules not processed for user `4645272c6f3849ffabd98634a927a461`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:10.661640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a54bc946-4d70-4e10-9c33-d409d1b64b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:10.736550 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d99f0-d7b2-45a5-b04f-3e2ed26d4822 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:10.752897 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b9515-414f-4001-b0ff-999421382831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:10.755948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b9515-414f-4001-b0ff-999421382831 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:10.757057 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b9515-414f-4001-b0ff-999421382831 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:10.758741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b9515-414f-4001-b0ff-999421382831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:10.784420 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b9515-414f-4001-b0ff-999421382831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:10.789394 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b9515-414f-4001-b0ff-999421382831 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:10.811421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 800/1613] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:10.813733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90cc32-1525-4a67-ac6a-6e1c2568f8e7 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:10.814325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90cc32-1525-4a67-ac6a-6e1c2568f8e7 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:10.814410 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90cc32-1525-4a67-ac6a-6e1c2568f8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:10.864062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ff90cc32-1525-4a67-ac6a-6e1c2568f8e7 None None] MFA Rules not processed for user `2080b428e34c491d92e7df054c98105b`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:10.865496 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ff90cc32-1525-4a67-ac6a-6e1c2568f8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:10.868234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 814/1614] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:10 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:10.869067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bc946-4d70-4e10-9c33-d409d1b64b72 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:10.897529 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bc946-4d70-4e10-9c33-d409d1b64b72 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:10.913331 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f392c48-36cc-441e-b6df-3af0aa79df99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:10.913966 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f392c48-36cc-441e-b6df-3af0aa79df99 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:10.914494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f392c48-36cc-441e-b6df-3af0aa79df99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:10.914772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f392c48-36cc-441e-b6df-3af0aa79df99 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:10.915517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f392c48-36cc-441e-b6df-3af0aa79df99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:10.918846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f392c48-36cc-441e-b6df-3af0aa79df99 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:10.975279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:10.982442 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 801/1615] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:10.990386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90cc32-1525-4a67-ac6a-6e1c2568f8e7 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:10.992655 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ff90cc32-1525-4a67-ac6a-6e1c2568f8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:11.021007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 815/1616] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:10 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:11.022837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90cc32-1525-4a67-ac6a-6e1c2568f8e7 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:11.023956 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f392c48-36cc-441e-b6df-3af0aa79df99 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:11.145591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375929b7-42b8-427c-92b9-7a8d6ed9eb8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:11.146579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375929b7-42b8-427c-92b9-7a8d6ed9eb8b service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:11.147818 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375929b7-42b8-427c-92b9-7a8d6ed9eb8b service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:11.148301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375929b7-42b8-427c-92b9-7a8d6ed9eb8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:11.174227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f392c48-36cc-441e-b6df-3af0aa79df99 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:11.195380 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c6962-1fb2-49af-bc33-e905d206454b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:11.196120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c6962-1fb2-49af-bc33-e905d206454b admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:11.196492 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c6962-1fb2-49af-bc33-e905d206454b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:11.196768 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c6962-1fb2-49af-bc33-e905d206454b admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:11.197549 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c6962-1fb2-49af-bc33-e905d206454b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:11.199608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c6962-1fb2-49af-bc33-e905d206454b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:11.226229 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:11.270432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 816/1617] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:11 2022] POST /identity/v3/projects => generated 403 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:11.270432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c6962-1fb2-49af-bc33-e905d206454b admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:11.332966 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375929b7-42b8-427c-92b9-7a8d6ed9eb8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:11.339375 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375929b7-42b8-427c-92b9-7a8d6ed9eb8b service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:11.369771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 802/1618] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:11.371141 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d2608-2544-46b6-947f-a70c2768ccb3 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:11.371427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d2608-2544-46b6-947f-a70c2768ccb3 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:11.371702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d2608-2544-46b6-947f-a70c2768ccb3 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:11.376066 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 803/1619] 10.222.0.69 () {54 vars in 962 bytes} [Wed Aug 3 18:50:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:50:11.378972 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d2608-2544-46b6-947f-a70c2768ccb3 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:11.470172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c6962-1fb2-49af-bc33-e905d206454b admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:11.483580 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0bfb36-3e09-4c52-bbf6-3c63ef96abb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:11.484523 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bfb36-3e09-4c52-bbf6-3c63ef96abb0 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:11.484867 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bfb36-3e09-4c52-bbf6-3c63ef96abb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:11.485232 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bfb36-3e09-4c52-bbf6-3c63ef96abb0 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:11.485871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0bfb36-3e09-4c52-bbf6-3c63ef96abb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:11.489780 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0bfb36-3e09-4c52-bbf6-3c63ef96abb0 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:11.544645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 817/1620] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:11 2022] POST /identity/v3/users => generated 297 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:11.547679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd8499-f357-4697-9d0a-6dfef442b0dc None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:11.547929 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd8499-f357-4697-9d0a-6dfef442b0dc None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:11.548240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd8499-f357-4697-9d0a-6dfef442b0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:11.581522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d2608-2544-46b6-947f-a70c2768ccb3 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:11.589477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a538dec-780a-4271-9213-7fbe4e1fdaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:11.595284 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a538dec-780a-4271-9213-7fbe4e1fdaca admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:11.595735 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a538dec-780a-4271-9213-7fbe4e1fdaca admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:11.596098 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a538dec-780a-4271-9213-7fbe4e1fdaca admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:11.597189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a538dec-780a-4271-9213-7fbe4e1fdaca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:11.599325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a538dec-780a-4271-9213-7fbe4e1fdaca admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:11.641567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 804/1621] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:11 2022] POST /identity/v3/users => generated 308 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:11.642552 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a538dec-780a-4271-9213-7fbe4e1fdaca admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:11.646637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-61bd8499-f357-4697-9d0a-6dfef442b0dc None None] MFA Rules not processed for user `850a50cfe4394febbafb0908b08888b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:11.648004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-61bd8499-f357-4697-9d0a-6dfef442b0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:11.666392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a538dec-780a-4271-9213-7fbe4e1fdaca admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:11.677873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af258243-635b-4294-b837-56ca0891bf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:11.678517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af258243-635b-4294-b837-56ca0891bf76 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:11.678875 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af258243-635b-4294-b837-56ca0891bf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:11.679223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af258243-635b-4294-b837-56ca0891bf76 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:11.682727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af258243-635b-4294-b837-56ca0891bf76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:11.684605 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af258243-635b-4294-b837-56ca0891bf76 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:11.712461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 805/1622] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:11.713376 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af258243-635b-4294-b837-56ca0891bf76 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:11.732316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af258243-635b-4294-b837-56ca0891bf76 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:11.749308 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9172450-a667-415f-9479-011e2d54bb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:11.750769 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9172450-a667-415f-9479-011e2d54bb4a admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:11.751277 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9172450-a667-415f-9479-011e2d54bb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:11.751751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9172450-a667-415f-9479-011e2d54bb4a admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:11.752701 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9172450-a667-415f-9479-011e2d54bb4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:11.755139 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9172450-a667-415f-9479-011e2d54bb4a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:11.761997 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:11.783650 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 806/1623] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:11.783650 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9172450-a667-415f-9479-011e2d54bb4a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:11.806361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9172450-a667-415f-9479-011e2d54bb4a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:11.817957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 818/1624] 10.222.0.69 () {58 vars in 1059 bytes} [Wed Aug 3 18:50:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:11.820483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea70931-44d0-4cf6-8a2e-bdbb47690f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:11.821255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd8499-f357-4697-9d0a-6dfef442b0dc None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:11.821400 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea70931-44d0-4cf6-8a2e-bdbb47690f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:11.821751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea70931-44d0-4cf6-8a2e-bdbb47690f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:11.822117 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea70931-44d0-4cf6-8a2e-bdbb47690f14 admin admin] PATH_INFO: `/v3/projects/b42450eacc8f4716b764d203e27d987e/users/b5e2de8372be47af8c90f29dd5c6d9c5/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:11.838581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd8499-f357-4697-9d0a-6dfef442b0dc None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:11.849101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e48b9-0338-4c5b-8108-8f94f91c9ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:11.849827 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e48b9-0338-4c5b-8108-8f94f91c9ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:11.850111 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e48b9-0338-4c5b-8108-8f94f91c9ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:11.850775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e48b9-0338-4c5b-8108-8f94f91c9ba5 admin admin] PATH_INFO: `/v3/projects/7bd41163613648df9a2bfde3a24d713f/users/03047a08f92543fc9a2a61169497c0e2/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:11.867213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea70931-44d0-4cf6-8a2e-bdbb47690f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42450eacc8f4716b764d203e27d987e, user_id=b5e2de8372be47af8c90f29dd5c6d9c5, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:11.870068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea70931-44d0-4cf6-8a2e-bdbb47690f14 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:11.892213 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e48b9-0338-4c5b-8108-8f94f91c9ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd41163613648df9a2bfde3a24d713f, user_id=03047a08f92543fc9a2a61169497c0e2, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:11.896641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e48b9-0338-4c5b-8108-8f94f91c9ba5 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:11.901024 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 807/1625] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:11 2022] PUT /identity/v3/projects/b42450eacc8f4716b764d203e27d987e/users/b5e2de8372be47af8c90f29dd5c6d9c5/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:11.904330 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea70931-44d0-4cf6-8a2e-bdbb47690f14 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:11.936701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 819/1626] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:11 2022] PUT /identity/v3/projects/7bd41163613648df9a2bfde3a24d713f/users/03047a08f92543fc9a2a61169497c0e2/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:11.936701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e48b9-0338-4c5b-8108-8f94f91c9ba5 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:12.027745 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e48b9-0338-4c5b-8108-8f94f91c9ba5 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:12.036433 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae3869-44f6-4bd3-ad28-cbbc4673331e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:12.037341 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae3869-44f6-4bd3-ad28-cbbc4673331e admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:12.037634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae3869-44f6-4bd3-ad28-cbbc4673331e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:12.038457 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae3869-44f6-4bd3-ad28-cbbc4673331e admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:12.039504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae3869-44f6-4bd3-ad28-cbbc4673331e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:12.043670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae3869-44f6-4bd3-ad28-cbbc4673331e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:12.058994 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 820/1627] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:12.059885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae3869-44f6-4bd3-ad28-cbbc4673331e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:12.066526 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea70931-44d0-4cf6-8a2e-bdbb47690f14 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:12.087728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae3869-44f6-4bd3-ad28-cbbc4673331e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:12.091607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:12.099713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03f631-ef07-449e-a6da-886cd8629c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:12.100460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03f631-ef07-449e-a6da-886cd8629c58 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:12.100745 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03f631-ef07-449e-a6da-886cd8629c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:12.101052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03f631-ef07-449e-a6da-886cd8629c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:12.102622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03f631-ef07-449e-a6da-886cd8629c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:12.104629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03f631-ef07-449e-a6da-886cd8629c58 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:12.123045 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 821/1628] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:12.123045 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03f631-ef07-449e-a6da-886cd8629c58 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:12.139459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03f631-ef07-449e-a6da-886cd8629c58 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:12.146419 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4dd6b-9bbc-4ffe-879a-d0694840807d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:12.147510 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4dd6b-9bbc-4ffe-879a-d0694840807d admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:12.148383 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4dd6b-9bbc-4ffe-879a-d0694840807d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:12.148894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4dd6b-9bbc-4ffe-879a-d0694840807d admin admin] PATH_INFO: `/v3/projects/b42450eacc8f4716b764d203e27d987e/users/b5e2de8372be47af8c90f29dd5c6d9c5/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:12.152120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4dd6b-9bbc-4ffe-879a-d0694840807d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42450eacc8f4716b764d203e27d987e, user_id=b5e2de8372be47af8c90f29dd5c6d9c5, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:12.157638 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4dd6b-9bbc-4ffe-879a-d0694840807d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:12.184180 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 822/1629] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:12 2022] PUT /identity/v3/projects/b42450eacc8f4716b764d203e27d987e/users/b5e2de8372be47af8c90f29dd5c6d9c5/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:12.184180 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4dd6b-9bbc-4ffe-879a-d0694840807d admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:12.206736 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ba307-0463-4cfb-b45d-a2e15b2b9418 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:12.207647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ba307-0463-4cfb-b45d-a2e15b2b9418 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:12.208014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ba307-0463-4cfb-b45d-a2e15b2b9418 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:12.208445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ba307-0463-4cfb-b45d-a2e15b2b9418 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:12.348781 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ba307-0463-4cfb-b45d-a2e15b2b9418 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:12.350534 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4dd6b-9bbc-4ffe-879a-d0694840807d admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:12.352144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ba307-0463-4cfb-b45d-a2e15b2b9418 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:12.364615 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7276696-7e68-47b3-81c9-811bf72760c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:12.365435 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7276696-7e68-47b3-81c9-811bf72760c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:12.378666 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7276696-7e68-47b3-81c9-811bf72760c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:12.379710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7276696-7e68-47b3-81c9-811bf72760c7 admin admin] PATH_INFO: `/v3/projects/7bd41163613648df9a2bfde3a24d713f/users/03047a08f92543fc9a2a61169497c0e2/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:12.386129 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7276696-7e68-47b3-81c9-811bf72760c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd41163613648df9a2bfde3a24d713f, user_id=03047a08f92543fc9a2a61169497c0e2, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:12.388210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 808/1630] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:12.392890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9e220-d909-4e6c-989d-7350955ead18 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:12.393280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7276696-7e68-47b3-81c9-811bf72760c7 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:12.396041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9e220-d909-4e6c-989d-7350955ead18 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:12.396500 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9e220-d909-4e6c-989d-7350955ead18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:12.421215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 823/1631] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:12 2022] PUT /identity/v3/projects/7bd41163613648df9a2bfde3a24d713f/users/03047a08f92543fc9a2a61169497c0e2/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:12.432830 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4c74c-3a8e-4101-9f03-0c1faa208467 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:12.432830 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4c74c-3a8e-4101-9f03-0c1faa208467 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:12.432830 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4c74c-3a8e-4101-9f03-0c1faa208467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:12.470618 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-e6e9e220-d909-4e6c-989d-7350955ead18 None None] MFA Rules not processed for user `b5e2de8372be47af8c90f29dd5c6d9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:12.472449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e6e9e220-d909-4e6c-989d-7350955ead18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:12.534396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-4bb4c74c-3a8e-4101-9f03-0c1faa208467 None None] MFA Rules not processed for user `03047a08f92543fc9a2a61169497c0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:12.538406 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4bb4c74c-3a8e-4101-9f03-0c1faa208467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:12.620335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:12.705508 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 809/1632] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:12.719946 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9e220-d909-4e6c-989d-7350955ead18 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:12.750002 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 824/1633] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:12 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:50:12.769827 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4c74c-3a8e-4101-9f03-0c1faa208467 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:12.883316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:12.887719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9e220-d909-4e6c-989d-7350955ead18 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:12.910055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557d960-e106-4ff0-8a63-5e88091d16fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:12.910708 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557d960-e106-4ff0-8a63-5e88091d16fe service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:12.911477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557d960-e106-4ff0-8a63-5e88091d16fe service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:12.911477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557d960-e106-4ff0-8a63-5e88091d16fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:12.936693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557d960-e106-4ff0-8a63-5e88091d16fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:12.943540 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557d960-e106-4ff0-8a63-5e88091d16fe service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:12.948911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4c74c-3a8e-4101-9f03-0c1faa208467 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:12.955004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0d918-fc73-4d32-8380-720a99c267ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:12.956124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0d918-fc73-4d32-8380-720a99c267ea service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:12.956774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0d918-fc73-4d32-8380-720a99c267ea service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:12.957255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0d918-fc73-4d32-8380-720a99c267ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:12.969046 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 810/1634] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:12.989680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0d918-fc73-4d32-8380-720a99c267ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:12.996647 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0d918-fc73-4d32-8380-720a99c267ea service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:13.049873 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 825/1635] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:12 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:50:13.370253 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557d960-e106-4ff0-8a63-5e88091d16fe service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:13.377066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0d918-fc73-4d32-8380-720a99c267ea service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:13.389685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0d918-fc73-4d32-8380-720a99c267ea service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:13.408414 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a337c-0605-4ed4-8a25-baf284fa0e41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:13.409083 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a337c-0605-4ed4-8a25-baf284fa0e41 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:13.409340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a337c-0605-4ed4-8a25-baf284fa0e41 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:13.412157 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a337c-0605-4ed4-8a25-baf284fa0e41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:13.550191 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557d960-e106-4ff0-8a63-5e88091d16fe service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:13.564205 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671e70c-9c24-4dc5-9d98-0a9ad6abc1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:13.565327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671e70c-9c24-4dc5-9d98-0a9ad6abc1f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:13.565327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671e70c-9c24-4dc5-9d98-0a9ad6abc1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:13.565524 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671e70c-9c24-4dc5-9d98-0a9ad6abc1f5 admin admin] PATH_INFO: `/v3/users/4645272c6f3849ffabd98634a927a461` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:13.566978 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671e70c-9c24-4dc5-9d98-0a9ad6abc1f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4645272c6f3849ffabd98634a927a461)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:13.568449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: WARNING py.warnings [None req-f671e70c-9c24-4dc5-9d98-0a9ad6abc1f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 18:50:13.568449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: warnings.warn(msg) Aug 03 18:50:13.568449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: Aug 03 18:50:13.569670 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671e70c-9c24-4dc5-9d98-0a9ad6abc1f5 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:13.646418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a337c-0605-4ed4-8a25-baf284fa0e41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:13.647886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a337c-0605-4ed4-8a25-baf284fa0e41 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:13.674563 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 826/1636] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:13.725201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127417a4-e296-4ce0-afb6-37f2186c3911 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:13.725201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127417a4-e296-4ce0-afb6-37f2186c3911 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:13.725201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127417a4-e296-4ce0-afb6-37f2186c3911 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:13.725201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 827/1637] 10.222.0.69 () {54 vars in 962 bytes} [Wed Aug 3 18:50:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:50:13.756344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f671e70c-9c24-4dc5-9d98-0a9ad6abc1f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4645272c6f3849ffabd98634a927a461'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:13.757482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d50ff-5853-448e-b937-e16ea4a2487b None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:13.757770 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d50ff-5853-448e-b937-e16ea4a2487b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:13.758054 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d50ff-5853-448e-b937-e16ea4a2487b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:13.783201 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f671e70c-9c24-4dc5-9d98-0a9ad6abc1f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4645272c6f3849ffabd98634a927a461'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:13.823424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f671e70c-9c24-4dc5-9d98-0a9ad6abc1f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4645272c6f3849ffabd98634a927a461'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:13.828451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-ed5d50ff-5853-448e-b937-e16ea4a2487b None None] MFA Rules not processed for user `850a50cfe4394febbafb0908b08888b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:13.829765 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ed5d50ff-5853-448e-b937-e16ea4a2487b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:13.849269 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f671e70c-9c24-4dc5-9d98-0a9ad6abc1f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4645272c6f3849ffabd98634a927a461'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:13.850391 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f671e70c-9c24-4dc5-9d98-0a9ad6abc1f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4645272c6f3849ffabd98634a927a461'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:13.860490 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 811/1638] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:13 2022] DELETE /identity/v3/users/4645272c6f3849ffabd98634a927a461 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:13.862454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dc1af-067e-43f8-ac63-abcfde2f6f57 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:13.863156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dc1af-067e-43f8-ac63-abcfde2f6f57 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:13.863776 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dc1af-067e-43f8-ac63-abcfde2f6f57 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:13.871143 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 812/1639] 10.222.0.69 () {54 vars in 912 bytes} [Wed Aug 3 18:50:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:50:13.872859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4de6c4-48f0-4c92-9b04-f382d075d477 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:13.873408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4de6c4-48f0-4c92-9b04-f382d075d477 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:13.874007 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4de6c4-48f0-4c92-9b04-f382d075d477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:13.907378 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-be4de6c4-48f0-4c92-9b04-f382d075d477 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:13.910070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-be4de6c4-48f0-4c92-9b04-f382d075d477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:14.000577 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:14.010199 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:14.032679 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 813/1640] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:14.034041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0bb61-42ba-4e05-bfcb-56ba367559ee None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:14.034274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0bb61-42ba-4e05-bfcb-56ba367559ee None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:14.034553 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0bb61-42ba-4e05-bfcb-56ba367559ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:14.038891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 828/1641] 10.222.0.69 () {58 vars in 1059 bytes} [Wed Aug 3 18:50:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 03 18:50:14.047372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c440c39-4cdb-4fbd-aa5f-604ec7bdbab0 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:14.047681 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c440c39-4cdb-4fbd-aa5f-604ec7bdbab0 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:14.048096 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c440c39-4cdb-4fbd-aa5f-604ec7bdbab0 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:14.049894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 829/1642] 10.222.0.69 () {54 vars in 912 bytes} [Wed Aug 3 18:50:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:50:14.061371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb1207-5148-40f5-a951-7ec5d1906b06 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:14.061664 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb1207-5148-40f5-a951-7ec5d1906b06 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:14.062554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb1207-5148-40f5-a951-7ec5d1906b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:14.084263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-b6f0bb61-42ba-4e05-bfcb-56ba367559ee None None] MFA Rules not processed for user `850a50cfe4394febbafb0908b08888b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:14.085529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b6f0bb61-42ba-4e05-bfcb-56ba367559ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:14.126095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-baeb1207-5148-40f5-a951-7ec5d1906b06 None None] MFA Rules not processed for user `850a50cfe4394febbafb0908b08888b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:14.129853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-baeb1207-5148-40f5-a951-7ec5d1906b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:14.264755 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 814/1643] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Aug 3 18:50:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:14.266114 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0bb61-42ba-4e05-bfcb-56ba367559ee None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:14.270934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b6f0bb61-42ba-4e05-bfcb-56ba367559ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:14.406937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 830/1644] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Aug 3 18:50:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:14.408846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb1207-5148-40f5-a951-7ec5d1906b06 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:14.410691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-baeb1207-5148-40f5-a951-7ec5d1906b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:14.454659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0bb61-42ba-4e05-bfcb-56ba367559ee None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:14.464503 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa5af5-b992-4a35-ae34-0a706d6732b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:14.465081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa5af5-b992-4a35-ae34-0a706d6732b3 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:14.465372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa5af5-b992-4a35-ae34-0a706d6732b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:14.465649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa5af5-b992-4a35-ae34-0a706d6732b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:14.494512 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa5af5-b992-4a35-ae34-0a706d6732b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:14.496198 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa5af5-b992-4a35-ae34-0a706d6732b3 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:14.516365 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb1207-5148-40f5-a951-7ec5d1906b06 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:14.520747 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 815/1645] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:14.521586 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa5af5-b992-4a35-ae34-0a706d6732b3 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:14.522885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-2eaa5af5-b992-4a35-ae34-0a706d6732b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:14.523140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408953e-cd05-4aa7-8ea0-d2ebb4fb1eb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:14.525476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408953e-cd05-4aa7-8ea0-d2ebb4fb1eb4 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:14.526248 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408953e-cd05-4aa7-8ea0-d2ebb4fb1eb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:14.526952 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408953e-cd05-4aa7-8ea0-d2ebb4fb1eb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:14.539174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa5af5-b992-4a35-ae34-0a706d6732b3 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:14.543737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a68e98-3c70-4b3f-a026-4df2aef6d273 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:14.544216 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a68e98-3c70-4b3f-a026-4df2aef6d273 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:14.544531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a68e98-3c70-4b3f-a026-4df2aef6d273 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:14.544856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a68e98-3c70-4b3f-a026-4df2aef6d273 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:14.558424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a68e98-3c70-4b3f-a026-4df2aef6d273 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:14.559523 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408953e-cd05-4aa7-8ea0-d2ebb4fb1eb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:14.559912 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a68e98-3c70-4b3f-a026-4df2aef6d273 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:14.561422 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408953e-cd05-4aa7-8ea0-d2ebb4fb1eb4 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:14.578630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 816/1646] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:14.586996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 831/1647] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:14.837427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a68e98-3c70-4b3f-a026-4df2aef6d273 service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:14.839335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-91a68e98-3c70-4b3f-a026-4df2aef6d273 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:14.868810 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a68e98-3c70-4b3f-a026-4df2aef6d273 service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:14.887660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23753e7-ea54-42d1-a352-d22ce317a16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:14.888952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23753e7-ea54-42d1-a352-d22ce317a16c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:14.889421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23753e7-ea54-42d1-a352-d22ce317a16c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:14.890452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23753e7-ea54-42d1-a352-d22ce317a16c admin admin] PATH_INFO: `/v3/users/2080b428e34c491d92e7df054c98105b` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:14.903239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23753e7-ea54-42d1-a352-d22ce317a16c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2080b428e34c491d92e7df054c98105b)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:14.912324 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23753e7-ea54-42d1-a352-d22ce317a16c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:15.030372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-a23753e7-ea54-42d1-a352-d22ce317a16c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2080b428e34c491d92e7df054c98105b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:15.045494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-a23753e7-ea54-42d1-a352-d22ce317a16c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2080b428e34c491d92e7df054c98105b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:15.075096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-a23753e7-ea54-42d1-a352-d22ce317a16c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2080b428e34c491d92e7df054c98105b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:15.083025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:15.095823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-a23753e7-ea54-42d1-a352-d22ce317a16c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2080b428e34c491d92e7df054c98105b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:15.097399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-a23753e7-ea54-42d1-a352-d22ce317a16c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2080b428e34c491d92e7df054c98105b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:15.119860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7ab65-7f9e-46ed-9de0-684164f57c30 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:15.120518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 817/1648] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:14 2022] DELETE /identity/v3/users/2080b428e34c491d92e7df054c98105b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:15.120655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7ab65-7f9e-46ed-9de0-684164f57c30 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:15.120655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7ab65-7f9e-46ed-9de0-684164f57c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:15.182379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-3da7ab65-7f9e-46ed-9de0-684164f57c30 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:15.185480 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-3da7ab65-7f9e-46ed-9de0-684164f57c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:15.433263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 832/1649] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:50:15.478796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23753e7-ea54-42d1-a352-d22ce317a16c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:15.481452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a23753e7-ea54-42d1-a352-d22ce317a16c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:15.598331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23753e7-ea54-42d1-a352-d22ce317a16c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:15.604517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f34e79-45e1-457b-a03e-9c792ecd2506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:15.642325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f34e79-45e1-457b-a03e-9c792ecd2506 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:15.642325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f34e79-45e1-457b-a03e-9c792ecd2506 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:15.642325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f34e79-45e1-457b-a03e-9c792ecd2506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:15.642325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f34e79-45e1-457b-a03e-9c792ecd2506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:15.642325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f34e79-45e1-457b-a03e-9c792ecd2506 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:15.690363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 818/1650] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:17.847297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7ab65-7f9e-46ed-9de0-684164f57c30 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:17.855263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-3da7ab65-7f9e-46ed-9de0-684164f57c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:17.891896 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7ab65-7f9e-46ed-9de0-684164f57c30 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:17.933607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:17.934821 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:17.935285 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:17.935772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] PATH_INFO: `/v3/projects/519094d345e8478bb11fd546fa97bdb2` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:17.940130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=519094d345e8478bb11fd546fa97bdb2)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:17.941421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: WARNING py.warnings [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 18:50:17.941421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: warnings.warn(msg) Aug 03 18:50:17.941421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: Aug 03 18:50:17.943004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:18.021109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:18.097863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '519094d345e8478bb11fd546fa97bdb2', 'request_id': 'req-eefc89cd-ef26-411b-9472-61dacd81fd6a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:18.119347 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '519094d345e8478bb11fd546fa97bdb2', 'request_id': 'req-eefc89cd-ef26-411b-9472-61dacd81fd6a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:18.133070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '519094d345e8478bb11fd546fa97bdb2', 'request_id': 'req-eefc89cd-ef26-411b-9472-61dacd81fd6a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:18.169262 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] The token cache is being invalidate because project 519094d345e8478bb11fd546fa97bdb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:18.169519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:18.174876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:18.183968 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 833/1651] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:17 2022] DELETE /identity/v3/projects/519094d345e8478bb11fd546fa97bdb2 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:18.188844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f34e79-45e1-457b-a03e-9c792ecd2506 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:18.201568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f34e79-45e1-457b-a03e-9c792ecd2506 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:18.403307 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0261d-ab41-4c5b-9687-3cb6bdfd3091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:18.403968 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0261d-ab41-4c5b-9687-3cb6bdfd3091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:18.404255 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0261d-ab41-4c5b-9687-3cb6bdfd3091 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:18.404587 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0261d-ab41-4c5b-9687-3cb6bdfd3091 admin admin] PATH_INFO: `/v3/users/03047a08f92543fc9a2a61169497c0e2` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:18.408614 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0261d-ab41-4c5b-9687-3cb6bdfd3091 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03047a08f92543fc9a2a61169497c0e2)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:18.410581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0261d-ab41-4c5b-9687-3cb6bdfd3091 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:18.570785 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c8c0261d-ab41-4c5b-9687-3cb6bdfd3091 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03047a08f92543fc9a2a61169497c0e2'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:18.584223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c8c0261d-ab41-4c5b-9687-3cb6bdfd3091 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03047a08f92543fc9a2a61169497c0e2'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:18.604199 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c8c0261d-ab41-4c5b-9687-3cb6bdfd3091 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03047a08f92543fc9a2a61169497c0e2'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:18.619248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c8c0261d-ab41-4c5b-9687-3cb6bdfd3091 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03047a08f92543fc9a2a61169497c0e2'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:18.620018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c8c0261d-ab41-4c5b-9687-3cb6bdfd3091 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03047a08f92543fc9a2a61169497c0e2'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:18.629391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:18.631193 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:18.649445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 819/1652] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:18 2022] DELETE /identity/v3/users/03047a08f92543fc9a2a61169497c0e2 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:18.654539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc89cd-ef26-411b-9472-61dacd81fd6a admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:18.859717 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fc41a-d157-4ba9-a021-090663f63952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:18.860868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fc41a-d157-4ba9-a021-090663f63952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:18.861123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fc41a-d157-4ba9-a021-090663f63952 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:18.861438 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fc41a-d157-4ba9-a021-090663f63952 admin admin] PATH_INFO: `/v3/projects/8fa50cb14ff74c8c9ec93af52f98ae2b` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:18.873561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092fc41a-d157-4ba9-a021-090663f63952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa50cb14ff74c8c9ec93af52f98ae2b)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:18.876801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092fc41a-d157-4ba9-a021-090663f63952 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:19.053097 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-092fc41a-d157-4ba9-a021-090663f63952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa50cb14ff74c8c9ec93af52f98ae2b', 'request_id': 'req-092fc41a-d157-4ba9-a021-090663f63952'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:19.072736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-092fc41a-d157-4ba9-a021-090663f63952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa50cb14ff74c8c9ec93af52f98ae2b', 'request_id': 'req-092fc41a-d157-4ba9-a021-090663f63952'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:19.096369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-092fc41a-d157-4ba9-a021-090663f63952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa50cb14ff74c8c9ec93af52f98ae2b', 'request_id': 'req-092fc41a-d157-4ba9-a021-090663f63952'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:19.131290 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-092fc41a-d157-4ba9-a021-090663f63952 admin admin] The token cache is being invalidate because project 8fa50cb14ff74c8c9ec93af52f98ae2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:19.131542 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-092fc41a-d157-4ba9-a021-090663f63952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:19.132432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-092fc41a-d157-4ba9-a021-090663f63952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:19.135497 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 834/1653] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:18 2022] DELETE /identity/v3/projects/8fa50cb14ff74c8c9ec93af52f98ae2b => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:19.141257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5aad91-2c69-43c6-b56e-2d0e97803321 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:19.141536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5aad91-2c69-43c6-b56e-2d0e97803321 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:19.141752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5aad91-2c69-43c6-b56e-2d0e97803321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:19.193693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ed5aad91-2c69-43c6-b56e-2d0e97803321 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:19.194948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ed5aad91-2c69-43c6-b56e-2d0e97803321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:19.212375 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:19.365246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 820/1654] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:19.372934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c2a9-7158-428f-a70d-e72cf16399ba None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:19.373225 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c2a9-7158-428f-a70d-e72cf16399ba None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:19.373507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c2a9-7158-428f-a70d-e72cf16399ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:19.419666 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-ac45c2a9-7158-428f-a70d-e72cf16399ba None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:19.426568 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ac45c2a9-7158-428f-a70d-e72cf16399ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:19.454421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 835/1655] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:19.459248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5aad91-2c69-43c6-b56e-2d0e97803321 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:19.472468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5aad91-2c69-43c6-b56e-2d0e97803321 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:19.480900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f67699-977b-480f-91c1-d254ab9593a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:19.483557 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f67699-977b-480f-91c1-d254ab9593a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:19.483896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f67699-977b-480f-91c1-d254ab9593a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:19.484173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f67699-977b-480f-91c1-d254ab9593a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:19.484959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f67699-977b-480f-91c1-d254ab9593a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:19.485322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f67699-977b-480f-91c1-d254ab9593a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:19.488167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f67699-977b-480f-91c1-d254ab9593a9 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:19.516990 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 821/1656] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:19.520937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45c2a9-7158-428f-a70d-e72cf16399ba None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:19.537137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45c2a9-7158-428f-a70d-e72cf16399ba None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:19.548464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298b798-bc2b-4779-858c-326ee037ffa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:19.549400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298b798-bc2b-4779-858c-326ee037ffa9 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:19.549823 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298b798-bc2b-4779-858c-326ee037ffa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:19.550238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298b798-bc2b-4779-858c-326ee037ffa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:19.551071 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298b798-bc2b-4779-858c-326ee037ffa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:19.553061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298b798-bc2b-4779-858c-326ee037ffa9 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:19.622153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 836/1657] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:19 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:19.627084 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f67699-977b-480f-91c1-d254ab9593a9 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:19.639179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f67699-977b-480f-91c1-d254ab9593a9 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:19.788423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011517bf-5cc7-4cb4-8c88-fc721ee3ea80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:19.790539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011517bf-5cc7-4cb4-8c88-fc721ee3ea80 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:19.790539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011517bf-5cc7-4cb4-8c88-fc721ee3ea80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:19.790539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011517bf-5cc7-4cb4-8c88-fc721ee3ea80 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:19.791641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011517bf-5cc7-4cb4-8c88-fc721ee3ea80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:19.794164 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011517bf-5cc7-4cb4-8c88-fc721ee3ea80 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:19.828037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 822/1658] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:19 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:50:19.829551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298b798-bc2b-4779-858c-326ee037ffa9 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:19.857584 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298b798-bc2b-4779-858c-326ee037ffa9 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:19.864679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f654a-2cfe-4970-91f5-5f6de91104fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:19.866315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f654a-2cfe-4970-91f5-5f6de91104fa admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:19.866987 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f654a-2cfe-4970-91f5-5f6de91104fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:19.867405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f654a-2cfe-4970-91f5-5f6de91104fa admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:19.868813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f654a-2cfe-4970-91f5-5f6de91104fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:19.872268 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f654a-2cfe-4970-91f5-5f6de91104fa admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:19.884343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:19.895722 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 837/1659] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:19.899875 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011517bf-5cc7-4cb4-8c88-fc721ee3ea80 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:19.937956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011517bf-5cc7-4cb4-8c88-fc721ee3ea80 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:19.950278 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552be2e-f88c-472a-a0df-d3c47795ea56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:19.951367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552be2e-f88c-472a-a0df-d3c47795ea56 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:19.951834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552be2e-f88c-472a-a0df-d3c47795ea56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:19.952432 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552be2e-f88c-472a-a0df-d3c47795ea56 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:19.954479 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552be2e-f88c-472a-a0df-d3c47795ea56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:19.959376 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552be2e-f88c-472a-a0df-d3c47795ea56 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:19.993065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 823/1660] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:19.994721 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f654a-2cfe-4970-91f5-5f6de91104fa admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:20.015711 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f654a-2cfe-4970-91f5-5f6de91104fa admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:20.036451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ec990-98d9-470b-95d6-13636c47835b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:20.037172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ec990-98d9-470b-95d6-13636c47835b admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:20.037457 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ec990-98d9-470b-95d6-13636c47835b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:20.037741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ec990-98d9-470b-95d6-13636c47835b admin admin] PATH_INFO: `/v3/projects/673c0edcde5a494190ffcc6025d77bcf/users/0cf71aba230e42d59178495387bacf49/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:20.104220 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ec990-98d9-470b-95d6-13636c47835b admin admin] RBAC: Authorizing `identity:create_grant(project_id=673c0edcde5a494190ffcc6025d77bcf, user_id=0cf71aba230e42d59178495387bacf49, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:20.110109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ec990-98d9-470b-95d6-13636c47835b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:20.141996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 838/1661] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:19 2022] PUT /identity/v3/projects/673c0edcde5a494190ffcc6025d77bcf/users/0cf71aba230e42d59178495387bacf49/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:20.152756 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114ef86-a8c3-4f1c-8a2e-237c5644388c None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:20.153034 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114ef86-a8c3-4f1c-8a2e-237c5644388c None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:20.153432 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114ef86-a8c3-4f1c-8a2e-237c5644388c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:20.224340 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-b114ef86-a8c3-4f1c-8a2e-237c5644388c None None] MFA Rules not processed for user `0cf71aba230e42d59178495387bacf49`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:20.242078 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b114ef86-a8c3-4f1c-8a2e-237c5644388c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:20.442180 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 824/1662] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:20.446517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ec990-98d9-470b-95d6-13636c47835b admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:20.598156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ec990-98d9-470b-95d6-13636c47835b admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:20.613116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47034f-b6d5-40b2-84e1-7a982f4e9154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:20.613838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47034f-b6d5-40b2-84e1-7a982f4e9154 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:20.614586 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47034f-b6d5-40b2-84e1-7a982f4e9154 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:20.614586 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47034f-b6d5-40b2-84e1-7a982f4e9154 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:20.615349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47034f-b6d5-40b2-84e1-7a982f4e9154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:20.617808 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47034f-b6d5-40b2-84e1-7a982f4e9154 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:20.700864 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 839/1663] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:20 2022] POST /identity/v3/projects => generated 373 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:20.702538 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114ef86-a8c3-4f1c-8a2e-237c5644388c None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:20.727570 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114ef86-a8c3-4f1c-8a2e-237c5644388c None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:20.737055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:20.948055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f728890-ecc8-422b-977f-46a60dc5317a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:20.950255 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f728890-ecc8-422b-977f-46a60dc5317a admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:20.951328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f728890-ecc8-422b-977f-46a60dc5317a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:20.953113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f728890-ecc8-422b-977f-46a60dc5317a admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:20.955091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f728890-ecc8-422b-977f-46a60dc5317a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:20.961515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f728890-ecc8-422b-977f-46a60dc5317a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:21.029382 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 825/1664] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:20 2022] POST /identity/v3/users => generated 293 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:21.031103 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47034f-b6d5-40b2-84e1-7a982f4e9154 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:21.046771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47034f-b6d5-40b2-84e1-7a982f4e9154 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:21.073969 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd791b21-4432-41cb-877b-f9d963275abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:21.080260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd791b21-4432-41cb-877b-f9d963275abe admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:21.080916 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd791b21-4432-41cb-877b-f9d963275abe admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:21.081789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd791b21-4432-41cb-877b-f9d963275abe admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:21.082655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd791b21-4432-41cb-877b-f9d963275abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:21.088164 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd791b21-4432-41cb-877b-f9d963275abe admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:21.116023 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 840/1665] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:21.120572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f728890-ecc8-422b-977f-46a60dc5317a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:21.138574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f728890-ecc8-422b-977f-46a60dc5317a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:21.155376 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a239e0-9e8c-4fae-a08d-fb8548843e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:21.158997 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a239e0-9e8c-4fae-a08d-fb8548843e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:21.159392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a239e0-9e8c-4fae-a08d-fb8548843e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:21.159778 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a239e0-9e8c-4fae-a08d-fb8548843e31 admin admin] PATH_INFO: `/v3/projects/5239e971491841b9b43abd6f6405468f/users/adb5e3da7b0a4744870f38831d0652dc/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:21.179681 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a239e0-9e8c-4fae-a08d-fb8548843e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5239e971491841b9b43abd6f6405468f, user_id=adb5e3da7b0a4744870f38831d0652dc, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:21.183244 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a239e0-9e8c-4fae-a08d-fb8548843e31 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:21.220368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 826/1666] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:21 2022] PUT /identity/v3/projects/5239e971491841b9b43abd6f6405468f/users/adb5e3da7b0a4744870f38831d0652dc/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:21.224334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd791b21-4432-41cb-877b-f9d963275abe admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:21.238153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd791b21-4432-41cb-877b-f9d963275abe admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:21.371276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e67f42-cd13-4810-9204-cb5c69490b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:21.372450 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e67f42-cd13-4810-9204-cb5c69490b72 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:21.372840 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e67f42-cd13-4810-9204-cb5c69490b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:21.373220 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e67f42-cd13-4810-9204-cb5c69490b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:21.375535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e67f42-cd13-4810-9204-cb5c69490b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:21.378760 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e67f42-cd13-4810-9204-cb5c69490b72 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:21.411352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 841/1667] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:21.415361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a239e0-9e8c-4fae-a08d-fb8548843e31 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:21.433677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a239e0-9e8c-4fae-a08d-fb8548843e31 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:21.456813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02741fc-00d3-4358-a29b-28cd63d0078c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:21.458767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02741fc-00d3-4358-a29b-28cd63d0078c admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:21.461862 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02741fc-00d3-4358-a29b-28cd63d0078c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:21.463737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02741fc-00d3-4358-a29b-28cd63d0078c admin admin] PATH_INFO: `/v3/projects/5239e971491841b9b43abd6f6405468f/users/adb5e3da7b0a4744870f38831d0652dc/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:21.467227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02741fc-00d3-4358-a29b-28cd63d0078c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5239e971491841b9b43abd6f6405468f, user_id=adb5e3da7b0a4744870f38831d0652dc, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:21.478896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02741fc-00d3-4358-a29b-28cd63d0078c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:21.515338 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 827/1668] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:21 2022] PUT /identity/v3/projects/5239e971491841b9b43abd6f6405468f/users/adb5e3da7b0a4744870f38831d0652dc/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:50:21.527411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e309449-d43f-42b7-bd68-a1ffe6ba5f66 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:21.527802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e309449-d43f-42b7-bd68-a1ffe6ba5f66 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:21.528150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e309449-d43f-42b7-bd68-a1ffe6ba5f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:21.598344 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-1e309449-d43f-42b7-bd68-a1ffe6ba5f66 None None] MFA Rules not processed for user `adb5e3da7b0a4744870f38831d0652dc`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:21.599683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1e309449-d43f-42b7-bd68-a1ffe6ba5f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:21.707410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:21.795105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 842/1669] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:21.807960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02741fc-00d3-4358-a29b-28cd63d0078c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:21.811675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d02741fc-00d3-4358-a29b-28cd63d0078c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:21.940932 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02741fc-00d3-4358-a29b-28cd63d0078c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:21.945172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba221e-26c3-49c0-8bed-20b91f381e86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:21.945810 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba221e-26c3-49c0-8bed-20b91f381e86 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:21.946100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba221e-26c3-49c0-8bed-20b91f381e86 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:21.946376 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba221e-26c3-49c0-8bed-20b91f381e86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:21.961825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba221e-26c3-49c0-8bed-20b91f381e86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:21.965042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba221e-26c3-49c0-8bed-20b91f381e86 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:21.988555 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 828/1670] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:22.024846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e309449-d43f-42b7-bd68-a1ffe6ba5f66 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:22.027095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1e309449-d43f-42b7-bd68-a1ffe6ba5f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:22.269182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e309449-d43f-42b7-bd68-a1ffe6ba5f66 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:22.279231 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15978ab8-611d-4ecd-9dc5-cb81f160811e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:22.279906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15978ab8-611d-4ecd-9dc5-cb81f160811e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:22.280384 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15978ab8-611d-4ecd-9dc5-cb81f160811e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:22.280750 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15978ab8-611d-4ecd-9dc5-cb81f160811e admin admin] PATH_INFO: `/v3/projects/7bd41163613648df9a2bfde3a24d713f` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:22.286762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15978ab8-611d-4ecd-9dc5-cb81f160811e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bd41163613648df9a2bfde3a24d713f)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:22.290177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15978ab8-611d-4ecd-9dc5-cb81f160811e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:22.444719 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-15978ab8-611d-4ecd-9dc5-cb81f160811e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bd41163613648df9a2bfde3a24d713f', 'request_id': 'req-15978ab8-611d-4ecd-9dc5-cb81f160811e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:22.454607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-15978ab8-611d-4ecd-9dc5-cb81f160811e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bd41163613648df9a2bfde3a24d713f', 'request_id': 'req-15978ab8-611d-4ecd-9dc5-cb81f160811e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:22.478410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-15978ab8-611d-4ecd-9dc5-cb81f160811e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bd41163613648df9a2bfde3a24d713f', 'request_id': 'req-15978ab8-611d-4ecd-9dc5-cb81f160811e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:22.532392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-15978ab8-611d-4ecd-9dc5-cb81f160811e admin admin] The token cache is being invalidate because project 7bd41163613648df9a2bfde3a24d713f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:22.532725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-15978ab8-611d-4ecd-9dc5-cb81f160811e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:22.536763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-15978ab8-611d-4ecd-9dc5-cb81f160811e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:22.540873 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 843/1671] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:22 2022] DELETE /identity/v3/projects/7bd41163613648df9a2bfde3a24d713f => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:22.550869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bb7ab-3089-42d4-b24b-4dbb36949532 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:22.551704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bb7ab-3089-42d4-b24b-4dbb36949532 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:22.552179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bb7ab-3089-42d4-b24b-4dbb36949532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:22.604967 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-3d4bb7ab-3089-42d4-b24b-4dbb36949532 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:22.610171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-3d4bb7ab-3089-42d4-b24b-4dbb36949532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:22.803090 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 829/1672] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:22.811048 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d3692-c2a6-4a9f-aa7a-d2cacc2bc11a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:22.811340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d3692-c2a6-4a9f-aa7a-d2cacc2bc11a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:22.812141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d3692-c2a6-4a9f-aa7a-d2cacc2bc11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:22.878749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-7e1d3692-c2a6-4a9f-aa7a-d2cacc2bc11a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:22.880265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7e1d3692-c2a6-4a9f-aa7a-d2cacc2bc11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:22.913256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 844/1673] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:22.914549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bb7ab-3089-42d4-b24b-4dbb36949532 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:22.923593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:22.930329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bb7ab-3089-42d4-b24b-4dbb36949532 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:22.940959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b6e81-f203-4cd0-8c45-7430254892b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:22.941866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b6e81-f203-4cd0-8c45-7430254892b9 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:22.945441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d3692-c2a6-4a9f-aa7a-d2cacc2bc11a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:22.947345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7e1d3692-c2a6-4a9f-aa7a-d2cacc2bc11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:22.955065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b6e81-f203-4cd0-8c45-7430254892b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:22.955344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b6e81-f203-4cd0-8c45-7430254892b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:22.956124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b6e81-f203-4cd0-8c45-7430254892b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:22.961197 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b6e81-f203-4cd0-8c45-7430254892b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:22.962480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b6e81-f203-4cd0-8c45-7430254892b9 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:22.969819 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d3692-c2a6-4a9f-aa7a-d2cacc2bc11a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:22.988217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05bf4d-a772-457b-a9c4-57cdaff2159d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:22.988925 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05bf4d-a772-457b-a9c4-57cdaff2159d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:22.989211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05bf4d-a772-457b-a9c4-57cdaff2159d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:22.989485 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05bf4d-a772-457b-a9c4-57cdaff2159d admin admin] PATH_INFO: `/v3/users/0cf71aba230e42d59178495387bacf49` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:22.995137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05bf4d-a772-457b-a9c4-57cdaff2159d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cf71aba230e42d59178495387bacf49)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:23.000046 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 830/1674] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:23.001176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05bf4d-a772-457b-a9c4-57cdaff2159d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:23.002045 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c13d4-e004-40c4-8266-c2dafdc1c9d2 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:23.002163 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c13d4-e004-40c4-8266-c2dafdc1c9d2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:23.004525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c13d4-e004-40c4-8266-c2dafdc1c9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:23.083637 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-c02c13d4-e004-40c4-8266-c2dafdc1c9d2 None None] MFA Rules not processed for user `c6085ba9dff1480a8a1674e492e0c6c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:23.085699 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c02c13d4-e004-40c4-8266-c2dafdc1c9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:23.099718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-fa05bf4d-a772-457b-a9c4-57cdaff2159d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cf71aba230e42d59178495387bacf49'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:23.103379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:23.116447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-fa05bf4d-a772-457b-a9c4-57cdaff2159d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cf71aba230e42d59178495387bacf49'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:23.200095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-fa05bf4d-a772-457b-a9c4-57cdaff2159d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cf71aba230e42d59178495387bacf49'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:23.212336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-fa05bf4d-a772-457b-a9c4-57cdaff2159d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cf71aba230e42d59178495387bacf49'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:23.215490 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-fa05bf4d-a772-457b-a9c4-57cdaff2159d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cf71aba230e42d59178495387bacf49'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:23.227642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 845/1675] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:22 2022] DELETE /identity/v3/users/0cf71aba230e42d59178495387bacf49 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:23.228665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05bf4d-a772-457b-a9c4-57cdaff2159d admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:23.230156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-fa05bf4d-a772-457b-a9c4-57cdaff2159d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:23.270738 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 831/1676] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:23.272593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa9ad2-805e-4e4c-bfcb-c06b620d443b None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:23.273119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa9ad2-805e-4e4c-bfcb-c06b620d443b None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:23.273472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa9ad2-805e-4e4c-bfcb-c06b620d443b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:23.308044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-39aa9ad2-805e-4e4c-bfcb-c06b620d443b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:23.308714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-39aa9ad2-805e-4e4c-bfcb-c06b620d443b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:23.373582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05bf4d-a772-457b-a9c4-57cdaff2159d admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:23.381843 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a57d6d-f1a9-4172-8b3e-afd4270b450d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:23.382507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a57d6d-f1a9-4172-8b3e-afd4270b450d admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:23.383627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a57d6d-f1a9-4172-8b3e-afd4270b450d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:23.383913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a57d6d-f1a9-4172-8b3e-afd4270b450d admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:23.384652 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a57d6d-f1a9-4172-8b3e-afd4270b450d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:23.386762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a57d6d-f1a9-4172-8b3e-afd4270b450d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:23.447610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 846/1677] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:23 2022] POST /identity/v3/projects => generated 405 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:23.449346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd897a-79a2-445e-b436-1b931f546156 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:23.449690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd897a-79a2-445e-b436-1b931f546156 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:23.450792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd897a-79a2-445e-b436-1b931f546156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:23.458830 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 832/1678] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:23.463081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa9ad2-805e-4e4c-bfcb-c06b620d443b None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:23.485593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa9ad2-805e-4e4c-bfcb-c06b620d443b None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:23.487364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-c0cd897a-79a2-445e-b436-1b931f546156 None None] MFA Rules not processed for user `c6085ba9dff1480a8a1674e492e0c6c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:23.489059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c0cd897a-79a2-445e-b436-1b931f546156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:23.569951 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 847/1679] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:23.582931 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd897a-79a2-445e-b436-1b931f546156 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:23.585296 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c0cd897a-79a2-445e-b436-1b931f546156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:23.618384 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9dba87-de19-4805-a956-aeef7c5aed7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:23.618935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9dba87-de19-4805-a956-aeef7c5aed7e admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:23.618935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9dba87-de19-4805-a956-aeef7c5aed7e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:23.619199 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9dba87-de19-4805-a956-aeef7c5aed7e admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:23.619939 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9dba87-de19-4805-a956-aeef7c5aed7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:23.623525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9dba87-de19-4805-a956-aeef7c5aed7e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:23.658239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 833/1680] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:23 2022] POST /identity/v3/users => generated 303 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:23.664576 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9dba87-de19-4805-a956-aeef7c5aed7e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:23.697234 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9dba87-de19-4805-a956-aeef7c5aed7e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:23.705890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc88ed-5364-4b4e-9d1d-6af22445b4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:23.706546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc88ed-5364-4b4e-9d1d-6af22445b4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:23.706833 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc88ed-5364-4b4e-9d1d-6af22445b4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:23.707102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc88ed-5364-4b4e-9d1d-6af22445b4a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:23.707808 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc88ed-5364-4b4e-9d1d-6af22445b4a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:23.709636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc88ed-5364-4b4e-9d1d-6af22445b4a3 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:23.718414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:23.729873 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:23.731383 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 834/1681] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:23.732012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc88ed-5364-4b4e-9d1d-6af22445b4a3 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:23.736206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-0fbc88ed-5364-4b4e-9d1d-6af22445b4a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:23.792493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd897a-79a2-445e-b436-1b931f546156 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:23.804918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c52231-9927-4a36-a0e4-ba11adc6bec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:23.807392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c52231-9927-4a36-a0e4-ba11adc6bec3 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:23.807937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c52231-9927-4a36-a0e4-ba11adc6bec3 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:23.808418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c52231-9927-4a36-a0e4-ba11adc6bec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:23.831165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c52231-9927-4a36-a0e4-ba11adc6bec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:23.834807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c52231-9927-4a36-a0e4-ba11adc6bec3 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:23.854078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 848/1682] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:50:23.856651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c52231-9927-4a36-a0e4-ba11adc6bec3 service nova] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:23.861094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc88ed-5364-4b4e-9d1d-6af22445b4a3 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:23.875520 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930141fe-26ee-4c1e-bb06-bb2929080711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:23.877006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930141fe-26ee-4c1e-bb06-bb2929080711 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:23.877298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930141fe-26ee-4c1e-bb06-bb2929080711 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:23.877629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930141fe-26ee-4c1e-bb06-bb2929080711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:23.899059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c52231-9927-4a36-a0e4-ba11adc6bec3 service nova] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:23.908261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930141fe-26ee-4c1e-bb06-bb2929080711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:23.910385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930141fe-26ee-4c1e-bb06-bb2929080711 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:23.919942 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d5303-a81e-46fe-b606-fef7acab94de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:23.920619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d5303-a81e-46fe-b606-fef7acab94de admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:23.920904 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d5303-a81e-46fe-b606-fef7acab94de admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:23.921178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d5303-a81e-46fe-b606-fef7acab94de admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:23.922065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d5303-a81e-46fe-b606-fef7acab94de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:23.923884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d5303-a81e-46fe-b606-fef7acab94de admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:23.945405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 849/1683] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:23.960235 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d5303-a81e-46fe-b606-fef7acab94de admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:23.971251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 835/1684] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:23.982906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d5303-a81e-46fe-b606-fef7acab94de admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:23.991741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09d873-8a49-489b-b7fc-f64c948fb040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:23.992693 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09d873-8a49-489b-b7fc-f64c948fb040 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:23.992987 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09d873-8a49-489b-b7fc-f64c948fb040 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:23.993269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09d873-8a49-489b-b7fc-f64c948fb040 admin admin] PATH_INFO: `/v3/projects/1a2d5828956b436ea162b749e3298006/users/246430adfe1b4144a52c3aede7307353/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:24.032324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09d873-8a49-489b-b7fc-f64c948fb040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a2d5828956b436ea162b749e3298006, user_id=246430adfe1b4144a52c3aede7307353, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:24.036079 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09d873-8a49-489b-b7fc-f64c948fb040 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:24.078670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 850/1685] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:23 2022] PUT /identity/v3/projects/1a2d5828956b436ea162b749e3298006/users/246430adfe1b4144a52c3aede7307353/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:24.080885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab40e2e-d1b4-4cbc-9f0f-41245b6ad8fe None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:24.081327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab40e2e-d1b4-4cbc-9f0f-41245b6ad8fe None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:24.081583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab40e2e-d1b4-4cbc-9f0f-41245b6ad8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:24.142299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-1ab40e2e-d1b4-4cbc-9f0f-41245b6ad8fe None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:24.144017 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1ab40e2e-d1b4-4cbc-9f0f-41245b6ad8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:24.266917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 836/1686] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:24.315286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09d873-8a49-489b-b7fc-f64c948fb040 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:24.470468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09d873-8a49-489b-b7fc-f64c948fb040 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:24.479725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01e0e4-afc1-4b30-91e0-8d59a950f17f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:24.480721 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01e0e4-afc1-4b30-91e0-8d59a950f17f service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:24.481100 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01e0e4-afc1-4b30-91e0-8d59a950f17f service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:24.481471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01e0e4-afc1-4b30-91e0-8d59a950f17f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:24.502341 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01e0e4-afc1-4b30-91e0-8d59a950f17f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:24.504376 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01e0e4-afc1-4b30-91e0-8d59a950f17f service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:24.522593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 851/1687] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:25.120055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723ddbd-9dfe-430c-9d3d-fc930c76a46c None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:25.120597 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723ddbd-9dfe-430c-9d3d-fc930c76a46c None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:25.120699 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723ddbd-9dfe-430c-9d3d-fc930c76a46c None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:25.122383 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 837/1688] 10.222.0.69 () {54 vars in 912 bytes} [Wed Aug 3 18:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:50:25.130883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378b6c1-e50f-4c6c-a87a-179ec6cb0cee None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:25.131726 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378b6c1-e50f-4c6c-a87a-179ec6cb0cee None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:25.132865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378b6c1-e50f-4c6c-a87a-179ec6cb0cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:25.163610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723ddbd-9dfe-430c-9d3d-fc930c76a46c None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:25.165541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e723ddbd-9dfe-430c-9d3d-fc930c76a46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:25.213616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-c378b6c1-e50f-4c6c-a87a-179ec6cb0cee None None] MFA Rules not processed for user `0eb9ad672e5444098d71b7c9a27b72e8`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:25.215034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c378b6c1-e50f-4c6c-a87a-179ec6cb0cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:25.215898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723ddbd-9dfe-430c-9d3d-fc930c76a46c None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:25.223758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd5f55-e5f2-4e63-a39b-20e048a8bc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:25.225369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd5f55-e5f2-4e63-a39b-20e048a8bc38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:25.225990 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd5f55-e5f2-4e63-a39b-20e048a8bc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:25.226826 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd5f55-e5f2-4e63-a39b-20e048a8bc38 admin admin] PATH_INFO: `/v3/projects/673c0edcde5a494190ffcc6025d77bcf` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:25.228358 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 852/1689] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Aug 3 18:50:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:25.229330 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd5f55-e5f2-4e63-a39b-20e048a8bc38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=673c0edcde5a494190ffcc6025d77bcf)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:25.234448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd5f55-e5f2-4e63-a39b-20e048a8bc38 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:25.245147 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378b6c1-e50f-4c6c-a87a-179ec6cb0cee None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:25.247201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c378b6c1-e50f-4c6c-a87a-179ec6cb0cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:25.260811 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378b6c1-e50f-4c6c-a87a-179ec6cb0cee None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:25.267938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e7a43-25c9-4317-b08b-bcef78fbd23b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:25.268769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e7a43-25c9-4317-b08b-bcef78fbd23b service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:25.269119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e7a43-25c9-4317-b08b-bcef78fbd23b service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:25.269460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e7a43-25c9-4317-b08b-bcef78fbd23b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:25.301105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e7a43-25c9-4317-b08b-bcef78fbd23b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:25.302997 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e7a43-25c9-4317-b08b-bcef78fbd23b service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:25.314511 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 853/1690] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:25.370546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-6dcd5f55-e5f2-4e63-a39b-20e048a8bc38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '673c0edcde5a494190ffcc6025d77bcf', 'request_id': 'req-6dcd5f55-e5f2-4e63-a39b-20e048a8bc38'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:25.386932 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-6dcd5f55-e5f2-4e63-a39b-20e048a8bc38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '673c0edcde5a494190ffcc6025d77bcf', 'request_id': 'req-6dcd5f55-e5f2-4e63-a39b-20e048a8bc38'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:25.395774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-6dcd5f55-e5f2-4e63-a39b-20e048a8bc38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '673c0edcde5a494190ffcc6025d77bcf', 'request_id': 'req-6dcd5f55-e5f2-4e63-a39b-20e048a8bc38'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:25.436098 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-6dcd5f55-e5f2-4e63-a39b-20e048a8bc38 admin admin] The token cache is being invalidate because project 673c0edcde5a494190ffcc6025d77bcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:25.436684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-6dcd5f55-e5f2-4e63-a39b-20e048a8bc38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:25.439139 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-6dcd5f55-e5f2-4e63-a39b-20e048a8bc38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:25.441641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 838/1691] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:25 2022] DELETE /identity/v3/projects/673c0edcde5a494190ffcc6025d77bcf => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:25.445784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e7a43-25c9-4317-b08b-bcef78fbd23b service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:25.464435 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e7a43-25c9-4317-b08b-bcef78fbd23b service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:25.612399 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24030712-dc83-4c30-8fcf-9b88efb59989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:25.613057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24030712-dc83-4c30-8fcf-9b88efb59989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:25.613057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24030712-dc83-4c30-8fcf-9b88efb59989 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:25.613374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24030712-dc83-4c30-8fcf-9b88efb59989 admin admin] PATH_INFO: `/v3/users/adb5e3da7b0a4744870f38831d0652dc` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:25.618344 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24030712-dc83-4c30-8fcf-9b88efb59989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adb5e3da7b0a4744870f38831d0652dc)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:25.625737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24030712-dc83-4c30-8fcf-9b88efb59989 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:25.661210 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:25.721224 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-24030712-dc83-4c30-8fcf-9b88efb59989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adb5e3da7b0a4744870f38831d0652dc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:25.758885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8815c2-c104-4a01-aace-ebcf2272ba5d None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:25.759181 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8815c2-c104-4a01-aace-ebcf2272ba5d None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:25.759453 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8815c2-c104-4a01-aace-ebcf2272ba5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:25.763731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-24030712-dc83-4c30-8fcf-9b88efb59989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adb5e3da7b0a4744870f38831d0652dc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:25.813154 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-24030712-dc83-4c30-8fcf-9b88efb59989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adb5e3da7b0a4744870f38831d0652dc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:25.817941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ab8815c2-c104-4a01-aace-ebcf2272ba5d None None] MFA Rules not processed for user `1f99fcc5eddd45c89abca70f00ad6fed`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:25.819767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ab8815c2-c104-4a01-aace-ebcf2272ba5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:25.838089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-24030712-dc83-4c30-8fcf-9b88efb59989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adb5e3da7b0a4744870f38831d0652dc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:25.841599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-24030712-dc83-4c30-8fcf-9b88efb59989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adb5e3da7b0a4744870f38831d0652dc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:25.855224 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:25.868182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 854/1692] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:25 2022] DELETE /identity/v3/users/adb5e3da7b0a4744870f38831d0652dc => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:26.019631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 839/1693] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:50:26.032918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fcaad-9271-43e9-96a0-55f5202ad409 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:26.032918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fcaad-9271-43e9-96a0-55f5202ad409 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:26.032918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fcaad-9271-43e9-96a0-55f5202ad409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:26.073060 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-9b2fcaad-9271-43e9-96a0-55f5202ad409 None None] MFA Rules not processed for user `1f99fcc5eddd45c89abca70f00ad6fed`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:26.074327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-9b2fcaad-9271-43e9-96a0-55f5202ad409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:26.245978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 855/1694] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:26.266688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8815c2-c104-4a01-aace-ebcf2272ba5d None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:26.268991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ab8815c2-c104-4a01-aace-ebcf2272ba5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:26.345055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07540a15-1382-4d0e-a4c8-d8aece1e9f7c None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:26.345342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07540a15-1382-4d0e-a4c8-d8aece1e9f7c None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:26.345628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07540a15-1382-4d0e-a4c8-d8aece1e9f7c None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:26.347783 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 856/1695] 10.222.0.69 () {54 vars in 961 bytes} [Wed Aug 3 18:50:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:50:26.362182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc667998-81e5-4b4a-9557-0bbfb5e04d96 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:26.362610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc667998-81e5-4b4a-9557-0bbfb5e04d96 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:26.362893 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc667998-81e5-4b4a-9557-0bbfb5e04d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:26.415958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-cc667998-81e5-4b4a-9557-0bbfb5e04d96 None None] MFA Rules not processed for user `fea93ceb4cbc49c2b4b280a4ddc257b7`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:26.417999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-cc667998-81e5-4b4a-9557-0bbfb5e04d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:26.517153 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8815c2-c104-4a01-aace-ebcf2272ba5d None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:26.601223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 857/1696] 10.222.0.69 () {58 vars in 1058 bytes} [Wed Aug 3 18:50:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 03 18:50:26.611167 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc667998-81e5-4b4a-9557-0bbfb5e04d96 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:26.622274 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc667998-81e5-4b4a-9557-0bbfb5e04d96 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:26.633066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811ce3e-be30-4d63-8d00-00f2d0dee4c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:26.633514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811ce3e-be30-4d63-8d00-00f2d0dee4c7 service placement] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:26.634091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811ce3e-be30-4d63-8d00-00f2d0dee4c7 service placement] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:26.635371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811ce3e-be30-4d63-8d00-00f2d0dee4c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:26.641876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0811ce3e-be30-4d63-8d00-00f2d0dee4c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:26.690477 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811ce3e-be30-4d63-8d00-00f2d0dee4c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:26.694106 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811ce3e-be30-4d63-8d00-00f2d0dee4c7 service placement] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:26.740656 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 858/1697] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:26.744181 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0dcb75-1c69-4956-bcb8-605cf7e8bb48 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:26.745884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0dcb75-1c69-4956-bcb8-605cf7e8bb48 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:26.746739 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0dcb75-1c69-4956-bcb8-605cf7e8bb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:26.843911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7334673-907e-4ed4-b56b-20f2c31d6bbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:26.849277 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7334673-907e-4ed4-b56b-20f2c31d6bbf service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:26.849782 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7334673-907e-4ed4-b56b-20f2c31d6bbf service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:26.851239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7334673-907e-4ed4-b56b-20f2c31d6bbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:26.862542 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-ee0dcb75-1c69-4956-bcb8-605cf7e8bb48 None None] MFA Rules not processed for user `c6085ba9dff1480a8a1674e492e0c6c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:26.866720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ee0dcb75-1c69-4956-bcb8-605cf7e8bb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:26.896248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7334673-907e-4ed4-b56b-20f2c31d6bbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:26.903140 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7334673-907e-4ed4-b56b-20f2c31d6bbf service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:26.945939 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 840/1698] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:26.954184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7334673-907e-4ed4-b56b-20f2c31d6bbf service nova] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:26.957915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:26.958881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e7334673-907e-4ed4-b56b-20f2c31d6bbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:27.168690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 859/1699] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:27.218114 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:27.357973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7334673-907e-4ed4-b56b-20f2c31d6bbf service nova] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:27.380208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2150be-7573-45ab-8215-9acfa7cc17aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:27.380931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2150be-7573-45ab-8215-9acfa7cc17aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:27.381217 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2150be-7573-45ab-8215-9acfa7cc17aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:27.381506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2150be-7573-45ab-8215-9acfa7cc17aa admin admin] PATH_INFO: `/v3/projects/5239e971491841b9b43abd6f6405468f` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:27.392871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2150be-7573-45ab-8215-9acfa7cc17aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5239e971491841b9b43abd6f6405468f)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:27.398513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2150be-7573-45ab-8215-9acfa7cc17aa admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:27.606723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1e2150be-7573-45ab-8215-9acfa7cc17aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5239e971491841b9b43abd6f6405468f', 'request_id': 'req-1e2150be-7573-45ab-8215-9acfa7cc17aa'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:27.630561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1e2150be-7573-45ab-8215-9acfa7cc17aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5239e971491841b9b43abd6f6405468f', 'request_id': 'req-1e2150be-7573-45ab-8215-9acfa7cc17aa'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:27.652215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1e2150be-7573-45ab-8215-9acfa7cc17aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5239e971491841b9b43abd6f6405468f', 'request_id': 'req-1e2150be-7573-45ab-8215-9acfa7cc17aa'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:27.692707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1e2150be-7573-45ab-8215-9acfa7cc17aa admin admin] The token cache is being invalidate because project 5239e971491841b9b43abd6f6405468f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:27.693019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1e2150be-7573-45ab-8215-9acfa7cc17aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:27.694088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1e2150be-7573-45ab-8215-9acfa7cc17aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:27.699220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 841/1700] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:26 2022] DELETE /identity/v3/projects/5239e971491841b9b43abd6f6405468f => generated 0 bytes in 753 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:27.705863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106c41f-7cf8-4604-9519-c9d3a90a30f4 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:27.706402 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106c41f-7cf8-4604-9519-c9d3a90a30f4 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:27.706923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106c41f-7cf8-4604-9519-c9d3a90a30f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:27.793358 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-0106c41f-7cf8-4604-9519-c9d3a90a30f4 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:27.801425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0106c41f-7cf8-4604-9519-c9d3a90a30f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:27.983259 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 860/1701] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:27.990756 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bd210-48b8-4223-a112-73807464f13e None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:27.991066 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bd210-48b8-4223-a112-73807464f13e None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:27.991322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bd210-48b8-4223-a112-73807464f13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:28.076133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ee6bd210-48b8-4223-a112-73807464f13e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:28.078262 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ee6bd210-48b8-4223-a112-73807464f13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:28.122784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106c41f-7cf8-4604-9519-c9d3a90a30f4 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:28.129857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 842/1702] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:28.137505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106c41f-7cf8-4604-9519-c9d3a90a30f4 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:28.163416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa88bad-ae86-430d-917c-de570053fb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:28.171725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa88bad-ae86-430d-917c-de570053fb0e admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:28.172044 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa88bad-ae86-430d-917c-de570053fb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:28.179075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa88bad-ae86-430d-917c-de570053fb0e admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:28.180906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa88bad-ae86-430d-917c-de570053fb0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:28.183284 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa88bad-ae86-430d-917c-de570053fb0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:28.190699 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa88bad-ae86-430d-917c-de570053fb0e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:28.266089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 861/1703] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:28.272381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bd210-48b8-4223-a112-73807464f13e None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:28.290852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bd210-48b8-4223-a112-73807464f13e None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:28.312844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef75529-0c8f-405b-b740-58f50215a7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:28.315026 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef75529-0c8f-405b-b740-58f50215a7ca admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:28.318817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef75529-0c8f-405b-b740-58f50215a7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:28.319351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef75529-0c8f-405b-b740-58f50215a7ca admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:28.320249 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef75529-0c8f-405b-b740-58f50215a7ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:28.329759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef75529-0c8f-405b-b740-58f50215a7ca admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:28.433316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 843/1704] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:28 2022] POST /identity/v3/projects => generated 363 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:28.434222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa88bad-ae86-430d-917c-de570053fb0e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:28.457550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa88bad-ae86-430d-917c-de570053fb0e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:28.537365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b537b40-9262-4843-8ca4-9fe110c58437 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:28.537660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b537b40-9262-4843-8ca4-9fe110c58437 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:28.537940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b537b40-9262-4843-8ca4-9fe110c58437 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:28.539543 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 844/1705] 10.222.0.69 () {54 vars in 912 bytes} [Wed Aug 3 18:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:50:28.552746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8d23d-da3c-4f7d-89df-4a5b6942a676 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:28.553051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8d23d-da3c-4f7d-89df-4a5b6942a676 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:28.553319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8d23d-da3c-4f7d-89df-4a5b6942a676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:28.619647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-18a8d23d-da3c-4f7d-89df-4a5b6942a676 None None] MFA Rules not processed for user `0eb9ad672e5444098d71b7c9a27b72e8`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:28.620955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-18a8d23d-da3c-4f7d-89df-4a5b6942a676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:28.729667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31216399-8cb0-4b3e-abd6-6b32e8049aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:28.732113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31216399-8cb0-4b3e-abd6-6b32e8049aee admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:28.734119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31216399-8cb0-4b3e-abd6-6b32e8049aee admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:28.736927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31216399-8cb0-4b3e-abd6-6b32e8049aee admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:28.739576 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31216399-8cb0-4b3e-abd6-6b32e8049aee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:28.753082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31216399-8cb0-4b3e-abd6-6b32e8049aee admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:28.767627 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 845/1706] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Aug 3 18:50:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:50:28.769604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae534b5-b5f0-445c-a2f2-4485e19798c2 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:28.769879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae534b5-b5f0-445c-a2f2-4485e19798c2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:28.770118 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae534b5-b5f0-445c-a2f2-4485e19798c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:28.834036 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 862/1707] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:28 2022] POST /identity/v3/users => generated 282 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:28.837697 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31216399-8cb0-4b3e-abd6-6b32e8049aee admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:28.838386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-8ae534b5-b5f0-445c-a2f2-4485e19798c2 None None] MFA Rules not processed for user `13ab977f852a41e898af1ddfb67bad69`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:28.839842 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8ae534b5-b5f0-445c-a2f2-4485e19798c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:28.840683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-31216399-8cb0-4b3e-abd6-6b32e8049aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:28.893506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31216399-8cb0-4b3e-abd6-6b32e8049aee admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:28.923146 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318bef2-5fc6-4f49-a337-bfe3ed0f47c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:28.923821 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318bef2-5fc6-4f49-a337-bfe3ed0f47c1 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:28.924108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318bef2-5fc6-4f49-a337-bfe3ed0f47c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:28.926734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318bef2-5fc6-4f49-a337-bfe3ed0f47c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:28.947730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318bef2-5fc6-4f49-a337-bfe3ed0f47c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:28.947730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318bef2-5fc6-4f49-a337-bfe3ed0f47c1 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:28.972389 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 863/1708] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:28.977384 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318bef2-5fc6-4f49-a337-bfe3ed0f47c1 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:29.002406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:29.009856 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318bef2-5fc6-4f49-a337-bfe3ed0f47c1 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:29.028178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600d1fe-e0d3-42ee-a4df-315344bfbf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:29.031706 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600d1fe-e0d3-42ee-a4df-315344bfbf33 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:29.032027 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600d1fe-e0d3-42ee-a4df-315344bfbf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:29.032374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600d1fe-e0d3-42ee-a4df-315344bfbf33 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:29.035155 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600d1fe-e0d3-42ee-a4df-315344bfbf33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:29.038803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600d1fe-e0d3-42ee-a4df-315344bfbf33 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:29.104946 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 846/1709] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:29.107575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 864/1710] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:29.118498 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae534b5-b5f0-445c-a2f2-4485e19798c2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:29.169376 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae534b5-b5f0-445c-a2f2-4485e19798c2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:29.188828 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd1d7e-db1b-4284-8e36-30597c820847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:29.189357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd1d7e-db1b-4284-8e36-30597c820847 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:29.189644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd1d7e-db1b-4284-8e36-30597c820847 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:29.189921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd1d7e-db1b-4284-8e36-30597c820847 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:29.190647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd1d7e-db1b-4284-8e36-30597c820847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:29.194764 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd1d7e-db1b-4284-8e36-30597c820847 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:29.224971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 847/1711] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:29.229064 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600d1fe-e0d3-42ee-a4df-315344bfbf33 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:29.247752 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600d1fe-e0d3-42ee-a4df-315344bfbf33 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:29.267014 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419f8c6-b7cd-45a2-97c2-1a5e5f5cf4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:29.267686 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419f8c6-b7cd-45a2-97c2-1a5e5f5cf4eb admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:29.267971 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419f8c6-b7cd-45a2-97c2-1a5e5f5cf4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:29.268248 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419f8c6-b7cd-45a2-97c2-1a5e5f5cf4eb admin admin] PATH_INFO: `/v3/projects/95827f33b42c4789856e0918ebf02807/users/2850912c2ac6417793a987a15334db28/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:29.319131 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419f8c6-b7cd-45a2-97c2-1a5e5f5cf4eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=95827f33b42c4789856e0918ebf02807, user_id=2850912c2ac6417793a987a15334db28, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:29.321708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419f8c6-b7cd-45a2-97c2-1a5e5f5cf4eb admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:29.353073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd1d7e-db1b-4284-8e36-30597c820847 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:29.355579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e4fd1d7e-db1b-4284-8e36-30597c820847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:29.357043 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:29.376582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 865/1712] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:29 2022] PUT /identity/v3/projects/95827f33b42c4789856e0918ebf02807/users/2850912c2ac6417793a987a15334db28/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:50:29.378182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece3d04-35ae-4d23-961e-9285132e9e31 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:29.379744 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece3d04-35ae-4d23-961e-9285132e9e31 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:29.382131 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece3d04-35ae-4d23-961e-9285132e9e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:29.467597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-1ece3d04-35ae-4d23-961e-9285132e9e31 None None] MFA Rules not processed for user `2850912c2ac6417793a987a15334db28`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:29.469979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1ece3d04-35ae-4d23-961e-9285132e9e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:29.618875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 866/1713] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:29.626696 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece3d04-35ae-4d23-961e-9285132e9e31 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:29.632600 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1ece3d04-35ae-4d23-961e-9285132e9e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:29.684630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd1d7e-db1b-4284-8e36-30597c820847 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:29.979277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece3d04-35ae-4d23-961e-9285132e9e31 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:30.019003 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d2c2d-d800-46ea-bd78-97ae9bb07bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:30.020073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d2c2d-d800-46ea-bd78-97ae9bb07bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:30.020569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d2c2d-d800-46ea-bd78-97ae9bb07bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:30.021082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d2c2d-d800-46ea-bd78-97ae9bb07bfe admin admin] PATH_INFO: `/v3/users/e01aa12142eb49d78094cbc5c740d62c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:30.030572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d2c2d-d800-46ea-bd78-97ae9bb07bfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=e01aa12142eb49d78094cbc5c740d62c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:30.031812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcb45b-3bca-474f-be08-bffeec1f8c43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:30.033000 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d2c2d-d800-46ea-bd78-97ae9bb07bfe admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:30.033959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcb45b-3bca-474f-be08-bffeec1f8c43 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:30.035863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcb45b-3bca-474f-be08-bffeec1f8c43 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:30.036366 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcb45b-3bca-474f-be08-bffeec1f8c43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:30.207992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-fa9d2c2d-d800-46ea-bd78-97ae9bb07bfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e01aa12142eb49d78094cbc5c740d62c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:30.262002 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-fa9d2c2d-d800-46ea-bd78-97ae9bb07bfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e01aa12142eb49d78094cbc5c740d62c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:30.304766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-fa9d2c2d-d800-46ea-bd78-97ae9bb07bfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e01aa12142eb49d78094cbc5c740d62c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:30.338566 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-fa9d2c2d-d800-46ea-bd78-97ae9bb07bfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e01aa12142eb49d78094cbc5c740d62c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:30.340214 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-fa9d2c2d-d800-46ea-bd78-97ae9bb07bfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e01aa12142eb49d78094cbc5c740d62c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:30.359729 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcb45b-3bca-474f-be08-bffeec1f8c43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:30.360760 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 867/1714] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:29 2022] DELETE /identity/v3/users/e01aa12142eb49d78094cbc5c740d62c => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:30.362652 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcb45b-3bca-474f-be08-bffeec1f8c43 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:30.366798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d2c2d-d800-46ea-bd78-97ae9bb07bfe admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:30.371906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-fa9d2c2d-d800-46ea-bd78-97ae9bb07bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:30.408396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 848/1715] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 1058 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:50:30.410258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87491215-560e-436b-aa15-3f09ed21fd26 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:30.410258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87491215-560e-436b-aa15-3f09ed21fd26 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:30.415914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87491215-560e-436b-aa15-3f09ed21fd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:30.431024 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:30.486895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:30.598973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-87491215-560e-436b-aa15-3f09ed21fd26 None None] MFA Rules not processed for user `246430adfe1b4144a52c3aede7307353`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:30.600240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-87491215-560e-436b-aa15-3f09ed21fd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:30.637118 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d2c2d-d800-46ea-bd78-97ae9bb07bfe admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:30.658120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2850e6a-0ad1-4e1a-abcb-f7b9106c6a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:30.658689 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2850e6a-0ad1-4e1a-abcb-f7b9106c6a93 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:30.658974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2850e6a-0ad1-4e1a-abcb-f7b9106c6a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:30.661453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2850e6a-0ad1-4e1a-abcb-f7b9106c6a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:30.662162 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2850e6a-0ad1-4e1a-abcb-f7b9106c6a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:30.668711 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2850e6a-0ad1-4e1a-abcb-f7b9106c6a93 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:30.832046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 868/1716] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:30 2022] POST /identity/v3/projects => generated 365 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:30.833497 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2850e6a-0ad1-4e1a-abcb-f7b9106c6a93 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:30.841773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 849/1717] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:30 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:30.845359 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9a483-ccc3-4884-96eb-5f4de4cc359d None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:30.845359 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9a483-ccc3-4884-96eb-5f4de4cc359d None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:30.845359 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9a483-ccc3-4884-96eb-5f4de4cc359d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:30.846938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d2850e6a-0ad1-4e1a-abcb-f7b9106c6a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:30.896416 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-aae9a483-ccc3-4884-96eb-5f4de4cc359d None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:30.898970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-aae9a483-ccc3-4884-96eb-5f4de4cc359d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:31.090792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 850/1718] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:31.091954 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9a483-ccc3-4884-96eb-5f4de4cc359d None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:31.141921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9a483-ccc3-4884-96eb-5f4de4cc359d None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:31.164471 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1647c5-571c-44cb-8055-260bb1752d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:31.168613 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1647c5-571c-44cb-8055-260bb1752d48 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:31.168922 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1647c5-571c-44cb-8055-260bb1752d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:31.169216 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1647c5-571c-44cb-8055-260bb1752d48 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:31.169938 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1647c5-571c-44cb-8055-260bb1752d48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:31.176846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2850e6a-0ad1-4e1a-abcb-f7b9106c6a93 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:31.181167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1647c5-571c-44cb-8055-260bb1752d48 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:31.194104 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7c290-78a4-4b14-81f5-2e0c16ccced9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:31.198827 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7c290-78a4-4b14-81f5-2e0c16ccced9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:31.199408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7c290-78a4-4b14-81f5-2e0c16ccced9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:31.199701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7c290-78a4-4b14-81f5-2e0c16ccced9 admin admin] PATH_INFO: `/v3/users/beb3c5d849a743ce969fb5e570c9b94f` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:31.205824 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7c290-78a4-4b14-81f5-2e0c16ccced9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beb3c5d849a743ce969fb5e570c9b94f)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:31.212802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7c290-78a4-4b14-81f5-2e0c16ccced9 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:31.286319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 851/1719] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:31 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:31.287470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1647c5-571c-44cb-8055-260bb1752d48 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:31.290119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ee1647c5-571c-44cb-8055-260bb1752d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:31.334200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1647c5-571c-44cb-8055-260bb1752d48 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:31.367556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd226a91-6455-4e0e-a6eb-02dffce326f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:31.368522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd226a91-6455-4e0e-a6eb-02dffce326f2 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:31.368914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd226a91-6455-4e0e-a6eb-02dffce326f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:31.369416 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd226a91-6455-4e0e-a6eb-02dffce326f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:31.370386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd226a91-6455-4e0e-a6eb-02dffce326f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:31.372693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd226a91-6455-4e0e-a6eb-02dffce326f2 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:31.407854 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-6ba7c290-78a4-4b14-81f5-2e0c16ccced9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beb3c5d849a743ce969fb5e570c9b94f'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:31.440864 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-6ba7c290-78a4-4b14-81f5-2e0c16ccced9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beb3c5d849a743ce969fb5e570c9b94f'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:31.451864 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 852/1720] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:31 2022] POST /identity/v3/projects => generated 405 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:31.460105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd226a91-6455-4e0e-a6eb-02dffce326f2 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:31.464472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-6ba7c290-78a4-4b14-81f5-2e0c16ccced9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beb3c5d849a743ce969fb5e570c9b94f'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:31.466884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-dd226a91-6455-4e0e-a6eb-02dffce326f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:31.501917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-6ba7c290-78a4-4b14-81f5-2e0c16ccced9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beb3c5d849a743ce969fb5e570c9b94f'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:31.502996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-6ba7c290-78a4-4b14-81f5-2e0c16ccced9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beb3c5d849a743ce969fb5e570c9b94f'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:31.512791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 869/1721] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:30 2022] DELETE /identity/v3/users/beb3c5d849a743ce969fb5e570c9b94f => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:31.513788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7c290-78a4-4b14-81f5-2e0c16ccced9 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:31.516109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-6ba7c290-78a4-4b14-81f5-2e0c16ccced9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:31.565256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:31.719512 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd226a91-6455-4e0e-a6eb-02dffce326f2 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:31.750276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:31.755642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7c290-78a4-4b14-81f5-2e0c16ccced9 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:31.773579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639761c4-52d5-4a8a-88b5-93025037713d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:31.774562 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639761c4-52d5-4a8a-88b5-93025037713d admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:31.774949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639761c4-52d5-4a8a-88b5-93025037713d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:31.775400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639761c4-52d5-4a8a-88b5-93025037713d admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:31.776254 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639761c4-52d5-4a8a-88b5-93025037713d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:31.782679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639761c4-52d5-4a8a-88b5-93025037713d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:31.807238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 870/1722] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:31.810827 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c8334-1f7a-47c1-9ca9-89a84056533e None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:31.816093 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c8334-1f7a-47c1-9ca9-89a84056533e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:31.817303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c8334-1f7a-47c1-9ca9-89a84056533e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:31.875671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0fc64-c022-42ae-8c33-f017544a9a65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:31.876356 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0fc64-c022-42ae-8c33-f017544a9a65 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:31.876677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0fc64-c022-42ae-8c33-f017544a9a65 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:31.876947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0fc64-c022-42ae-8c33-f017544a9a65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:31.881948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-31f0fc64-c022-42ae-8c33-f017544a9a65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:31.906546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-873c8334-1f7a-47c1-9ca9-89a84056533e None None] MFA Rules not processed for user `1aa1d56d9b154e32aa1c136b577d49f4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:31.914893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0fc64-c022-42ae-8c33-f017544a9a65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:31.916774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0fc64-c022-42ae-8c33-f017544a9a65 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:31.922428 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-873c8334-1f7a-47c1-9ca9-89a84056533e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:31.944271 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 853/1723] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:31.952654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0fc64-c022-42ae-8c33-f017544a9a65 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:31.957788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-31f0fc64-c022-42ae-8c33-f017544a9a65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:31.986163 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0fc64-c022-42ae-8c33-f017544a9a65 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:32.003020 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35fe94-288e-4516-8c4e-e6295cc9401d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:32.003238 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35fe94-288e-4516-8c4e-e6295cc9401d admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:32.004066 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35fe94-288e-4516-8c4e-e6295cc9401d admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:32.004628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35fe94-288e-4516-8c4e-e6295cc9401d admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:32.005466 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35fe94-288e-4516-8c4e-e6295cc9401d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:32.008113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35fe94-288e-4516-8c4e-e6295cc9401d admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:32.083692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 854/1724] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:31 2022] POST /identity/v3/users => generated 309 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:32.086399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ff4de-8f66-4e8b-a2b4-0121545b5a9b None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:32.086698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ff4de-8f66-4e8b-a2b4-0121545b5a9b None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:32.089649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ff4de-8f66-4e8b-a2b4-0121545b5a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:32.107255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 871/1725] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:32.109374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aed54f-ea13-4731-a493-e62f0aa4092f None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:32.109668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aed54f-ea13-4731-a493-e62f0aa4092f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:32.110130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aed54f-ea13-4731-a493-e62f0aa4092f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:32.118150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-7b0ff4de-8f66-4e8b-a2b4-0121545b5a9b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:32.119982 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-7b0ff4de-8f66-4e8b-a2b4-0121545b5a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:32.132507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 855/1726] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:32.133745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ff4de-8f66-4e8b-a2b4-0121545b5a9b None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:32.146216 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ff4de-8f66-4e8b-a2b4-0121545b5a9b None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:32.153803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4821a8-026b-46ea-b910-f4af8569a832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:32.155041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4821a8-026b-46ea-b910-f4af8569a832 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:32.156058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4821a8-026b-46ea-b910-f4af8569a832 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:32.156640 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4821a8-026b-46ea-b910-f4af8569a832 admin admin] PATH_INFO: `/v3/projects/4a2a6ea216704e7eaf2aa8f68fedab9c/users/f396c8dc1a264966be71e4a865b30940/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:32.190417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4821a8-026b-46ea-b910-f4af8569a832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2a6ea216704e7eaf2aa8f68fedab9c, user_id=f396c8dc1a264966be71e4a865b30940, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:32.206215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4821a8-026b-46ea-b910-f4af8569a832 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:32.252875 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 856/1727] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:32 2022] PUT /identity/v3/projects/4a2a6ea216704e7eaf2aa8f68fedab9c/users/f396c8dc1a264966be71e4a865b30940/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:32.257988 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4821a8-026b-46ea-b910-f4af8569a832 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:32.278400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-16aed54f-ea13-4731-a493-e62f0aa4092f None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:32.284007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-16aed54f-ea13-4731-a493-e62f0aa4092f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:32.519306 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4821a8-026b-46ea-b910-f4af8569a832 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:32.538276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f339a-2c1b-4fae-9481-8a1487077f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:32.538950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f339a-2c1b-4fae-9481-8a1487077f52 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:32.539265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f339a-2c1b-4fae-9481-8a1487077f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:32.539586 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f339a-2c1b-4fae-9481-8a1487077f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:32.540352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f339a-2c1b-4fae-9481-8a1487077f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:32.541990 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f339a-2c1b-4fae-9481-8a1487077f52 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:32.555584 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 872/1728] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Aug 3 18:50:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:32.562003 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aed54f-ea13-4731-a493-e62f0aa4092f None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:32.565529 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-16aed54f-ea13-4731-a493-e62f0aa4092f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:32.589626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 857/1729] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:32.590543 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f339a-2c1b-4fae-9481-8a1487077f52 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:32.606030 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-0c8f339a-2c1b-4fae-9481-8a1487077f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:32.609533 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:32.810508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aed54f-ea13-4731-a493-e62f0aa4092f None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:32.814151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f339a-2c1b-4fae-9481-8a1487077f52 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:32.818766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92cf46-87ef-495b-9de6-9c3ddf57b00c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:32.819624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92cf46-87ef-495b-9de6-9c3ddf57b00c service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:32.819968 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92cf46-87ef-495b-9de6-9c3ddf57b00c service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:32.820261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92cf46-87ef-495b-9de6-9c3ddf57b00c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:32.825563 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d525437-7534-483a-8393-96f219371e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:32.827589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d525437-7534-483a-8393-96f219371e08 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:32.829701 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d525437-7534-483a-8393-96f219371e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:32.830025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d525437-7534-483a-8393-96f219371e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:32.856711 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d525437-7534-483a-8393-96f219371e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:32.860854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d525437-7534-483a-8393-96f219371e08 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:32.917230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 858/1730] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:32.918772 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d525437-7534-483a-8393-96f219371e08 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:32.927294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92cf46-87ef-495b-9de6-9c3ddf57b00c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:32.939635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:32.940872 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92cf46-87ef-495b-9de6-9c3ddf57b00c service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:32.953871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 873/1731] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:32.954871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92cf46-87ef-495b-9de6-9c3ddf57b00c service nova] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:32.955387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d525437-7534-483a-8393-96f219371e08 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:32.968023 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92cf46-87ef-495b-9de6-9c3ddf57b00c service nova] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:32.975924 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db592a-8f6d-4a6c-b023-52d4490d3fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:32.976984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd178a-0ae1-4dc9-af23-3f766d3f34a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:32.977556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db592a-8f6d-4a6c-b023-52d4490d3fa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:32.977907 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd178a-0ae1-4dc9-af23-3f766d3f34a5 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:32.978094 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db592a-8f6d-4a6c-b023-52d4490d3fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:32.978195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd178a-0ae1-4dc9-af23-3f766d3f34a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:32.978382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db592a-8f6d-4a6c-b023-52d4490d3fa9 admin admin] PATH_INFO: `/v3/projects/387e0625f71b4ace85f7659d71b3537f/users/c84338b02b474eaa9f02288d679da181/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:32.978481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd178a-0ae1-4dc9-af23-3f766d3f34a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:32.979154 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd178a-0ae1-4dc9-af23-3f766d3f34a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:32.987673 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd178a-0ae1-4dc9-af23-3f766d3f34a5 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:32.997240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db592a-8f6d-4a6c-b023-52d4490d3fa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387e0625f71b4ace85f7659d71b3537f, user_id=c84338b02b474eaa9f02288d679da181, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:33.001550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db592a-8f6d-4a6c-b023-52d4490d3fa9 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:33.034064 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 859/1732] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:33.035088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd178a-0ae1-4dc9-af23-3f766d3f34a5 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:33.041566 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c3bd178a-0ae1-4dc9-af23-3f766d3f34a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:33.053777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 874/1733] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:32 2022] PUT /identity/v3/projects/387e0625f71b4ace85f7659d71b3537f/users/c84338b02b474eaa9f02288d679da181/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:33.054973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db592a-8f6d-4a6c-b023-52d4490d3fa9 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:33.229066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db592a-8f6d-4a6c-b023-52d4490d3fa9 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:33.240200 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ab26f-b20e-492c-ad95-b7e7eac704cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:33.241174 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ab26f-b20e-492c-ad95-b7e7eac704cd admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:33.241174 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ab26f-b20e-492c-ad95-b7e7eac704cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:33.241456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ab26f-b20e-492c-ad95-b7e7eac704cd admin admin] PATH_INFO: `/v3/projects/4a2a6ea216704e7eaf2aa8f68fedab9c/users/f396c8dc1a264966be71e4a865b30940/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:33.246476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ab26f-b20e-492c-ad95-b7e7eac704cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2a6ea216704e7eaf2aa8f68fedab9c, user_id=f396c8dc1a264966be71e4a865b30940, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:33.253362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ab26f-b20e-492c-ad95-b7e7eac704cd admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:33.281114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 875/1734] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:33 2022] PUT /identity/v3/projects/4a2a6ea216704e7eaf2aa8f68fedab9c/users/f396c8dc1a264966be71e4a865b30940/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:33.284659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ab26f-b20e-492c-ad95-b7e7eac704cd admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:33.328826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:33.387461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd178a-0ae1-4dc9-af23-3f766d3f34a5 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:33.490127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ab26f-b20e-492c-ad95-b7e7eac704cd admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:33.538343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51cd194-1ffc-48dc-b5d7-17e0c105da8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:33.543313 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cd194-1ffc-48dc-b5d7-17e0c105da8b admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:33.545595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cd194-1ffc-48dc-b5d7-17e0c105da8b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:33.545595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cd194-1ffc-48dc-b5d7-17e0c105da8b admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:33.548965 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51cd194-1ffc-48dc-b5d7-17e0c105da8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:33.557861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51cd194-1ffc-48dc-b5d7-17e0c105da8b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:33.621818 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 876/1735] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:33.626623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e7a8f-eae5-4a5c-bd3c-71f7879be390 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:33.630367 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e7a8f-eae5-4a5c-bd3c-71f7879be390 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:33.630825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e7a8f-eae5-4a5c-bd3c-71f7879be390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:33.633891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfd70f-632a-4db3-90f6-608b0424f8c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:33.636744 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfd70f-632a-4db3-90f6-608b0424f8c0 service swift] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:33.637201 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfd70f-632a-4db3-90f6-608b0424f8c0 service swift] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:33.637487 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfd70f-632a-4db3-90f6-608b0424f8c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:33.755584 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-374e7a8f-eae5-4a5c-bd3c-71f7879be390 None None] MFA Rules not processed for user `f396c8dc1a264966be71e4a865b30940`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:33.757147 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-374e7a8f-eae5-4a5c-bd3c-71f7879be390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:33.844182 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dfd70f-632a-4db3-90f6-608b0424f8c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:33.852893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dfd70f-632a-4db3-90f6-608b0424f8c0 service swift] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:33.888897 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 860/1736] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 18:50:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 854 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 18:50:33.891646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfd70f-632a-4db3-90f6-608b0424f8c0 service swift] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:33.907295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfd70f-632a-4db3-90f6-608b0424f8c0 service swift] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:33.932343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3270db-a53e-497b-8aae-8a8042e7a2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:33.932343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3270db-a53e-497b-8aae-8a8042e7a2c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:33.932343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3270db-a53e-497b-8aae-8a8042e7a2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:33.932343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3270db-a53e-497b-8aae-8a8042e7a2c2 admin admin] PATH_INFO: `/v3/projects/387e0625f71b4ace85f7659d71b3537f/users/c84338b02b474eaa9f02288d679da181/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:33.939065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3270db-a53e-497b-8aae-8a8042e7a2c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387e0625f71b4ace85f7659d71b3537f, user_id=c84338b02b474eaa9f02288d679da181, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:33.945535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3270db-a53e-497b-8aae-8a8042e7a2c2 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:33.990105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 861/1737] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:33 2022] PUT /identity/v3/projects/387e0625f71b4ace85f7659d71b3537f/users/c84338b02b474eaa9f02288d679da181/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:34.026894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 877/1738] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:34.037061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3270db-a53e-497b-8aae-8a8042e7a2c2 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:34.040164 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-3c3270db-a53e-497b-8aae-8a8042e7a2c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:34.054513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:34.214264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3270db-a53e-497b-8aae-8a8042e7a2c2 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:34.254678 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9096e8-2a3e-41ed-9a23-348076b412a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:34.256659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9096e8-2a3e-41ed-9a23-348076b412a0 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:34.258095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9096e8-2a3e-41ed-9a23-348076b412a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:34.258387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9096e8-2a3e-41ed-9a23-348076b412a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:34.576193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9096e8-2a3e-41ed-9a23-348076b412a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:34.578908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9096e8-2a3e-41ed-9a23-348076b412a0 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:34.590015 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e7a8f-eae5-4a5c-bd3c-71f7879be390 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:34.592065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-374e7a8f-eae5-4a5c-bd3c-71f7879be390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:34.611113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 862/1739] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:34.650417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e7a8f-eae5-4a5c-bd3c-71f7879be390 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:34.834405 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9096e8-2a3e-41ed-9a23-348076b412a0 service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:34.851247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9096e8-2a3e-41ed-9a23-348076b412a0 service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:34.856510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907e5d2-f125-4af7-8736-1c2108eb931c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:34.857312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907e5d2-f125-4af7-8736-1c2108eb931c service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:34.857708 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907e5d2-f125-4af7-8736-1c2108eb931c service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:34.858671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907e5d2-f125-4af7-8736-1c2108eb931c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:34.861365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8907e5d2-f125-4af7-8736-1c2108eb931c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:34.867283 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702d5e3-22a1-45e0-937b-7990d3252479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:34.868188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702d5e3-22a1-45e0-937b-7990d3252479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:34.868641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702d5e3-22a1-45e0-937b-7990d3252479 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:34.869072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702d5e3-22a1-45e0-937b-7990d3252479 admin admin] PATH_INFO: `/v3/projects/b16c594fd667419ba6d04396b3c730ef` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:34.870831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702d5e3-22a1-45e0-937b-7990d3252479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b16c594fd667419ba6d04396b3c730ef)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:34.875575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702d5e3-22a1-45e0-937b-7990d3252479 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:34.980581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907e5d2-f125-4af7-8736-1c2108eb931c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:34.982667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907e5d2-f125-4af7-8736-1c2108eb931c service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:35.016072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:35.024952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 863/1740] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:35.055552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c702d5e3-22a1-45e0-937b-7990d3252479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b16c594fd667419ba6d04396b3c730ef', 'request_id': 'req-c702d5e3-22a1-45e0-937b-7990d3252479'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:35.090655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c702d5e3-22a1-45e0-937b-7990d3252479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b16c594fd667419ba6d04396b3c730ef', 'request_id': 'req-c702d5e3-22a1-45e0-937b-7990d3252479'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:35.116452 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c702d5e3-22a1-45e0-937b-7990d3252479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b16c594fd667419ba6d04396b3c730ef', 'request_id': 'req-c702d5e3-22a1-45e0-937b-7990d3252479'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:35.176045 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c702d5e3-22a1-45e0-937b-7990d3252479 admin admin] The token cache is being invalidate because project b16c594fd667419ba6d04396b3c730ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:35.177944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c702d5e3-22a1-45e0-937b-7990d3252479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:35.182082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c702d5e3-22a1-45e0-937b-7990d3252479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:35.194042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1ed17-4cf5-49f6-bf0a-5cd2c7c1a549 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:35.194625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1ed17-4cf5-49f6-bf0a-5cd2c7c1a549 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:35.195217 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1ed17-4cf5-49f6-bf0a-5cd2c7c1a549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:35.200678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 878/1741] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:34 2022] DELETE /identity/v3/projects/b16c594fd667419ba6d04396b3c730ef => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:50:35.207329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb6f13-2652-4800-8196-97e359000416 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:35.207916 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb6f13-2652-4800-8196-97e359000416 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:35.208747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb6f13-2652-4800-8196-97e359000416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:35.272484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-dfcb6f13-2652-4800-8196-97e359000416 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:35.273722 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-dfcb6f13-2652-4800-8196-97e359000416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:35.279276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-aad1ed17-4cf5-49f6-bf0a-5cd2c7c1a549 None None] MFA Rules not processed for user `0eb9ad672e5444098d71b7c9a27b72e8`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:35.280947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-aad1ed17-4cf5-49f6-bf0a-5cd2c7c1a549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:35.482675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 864/1742] 10.222.0.73 () {58 vars in 1015 bytes} [Wed Aug 3 18:50:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:35.493876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 879/1743] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:50:35.502723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0ed50-28ee-46ca-a26f-20dca175a3f4 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:35.503138 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0ed50-28ee-46ca-a26f-20dca175a3f4 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:35.503447 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0ed50-28ee-46ca-a26f-20dca175a3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:35.521243 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:35.553045 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-baa0ed50-28ee-46ca-a26f-20dca175a3f4 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:35.554305 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-baa0ed50-28ee-46ca-a26f-20dca175a3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:35.579128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 865/1744] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:35.582895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb6f13-2652-4800-8196-97e359000416 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:35.599785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb6f13-2652-4800-8196-97e359000416 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:35.610946 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0ed50-28ee-46ca-a26f-20dca175a3f4 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:35.612901 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-baa0ed50-28ee-46ca-a26f-20dca175a3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:35.614511 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9bd41-081a-4244-8c54-9d92fc5d002e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:35.615252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9bd41-081a-4244-8c54-9d92fc5d002e admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:35.615627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9bd41-081a-4244-8c54-9d92fc5d002e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:35.616635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9bd41-081a-4244-8c54-9d92fc5d002e admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:35.617421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9bd41-081a-4244-8c54-9d92fc5d002e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:35.617820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9bd41-081a-4244-8c54-9d92fc5d002e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:35.619513 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9bd41-081a-4244-8c54-9d92fc5d002e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:35.643209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0ed50-28ee-46ca-a26f-20dca175a3f4 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:35.654745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52d254-1b5d-4f77-9fd6-9fa264fe34f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:35.656459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52d254-1b5d-4f77-9fd6-9fa264fe34f6 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:35.656694 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 880/1745] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:35.657213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52d254-1b5d-4f77-9fd6-9fa264fe34f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:35.657602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52d254-1b5d-4f77-9fd6-9fa264fe34f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:35.660886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9bd41-081a-4244-8c54-9d92fc5d002e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:35.676849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9bd41-081a-4244-8c54-9d92fc5d002e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:35.678804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52d254-1b5d-4f77-9fd6-9fa264fe34f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:35.680280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52d254-1b5d-4f77-9fd6-9fa264fe34f6 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:35.688781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb88100-ce71-48a2-bf56-624f61f57c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:35.689432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb88100-ce71-48a2-bf56-624f61f57c99 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:35.689721 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb88100-ce71-48a2-bf56-624f61f57c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:35.690010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb88100-ce71-48a2-bf56-624f61f57c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:35.690561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 866/1746] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:50:35.690877 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb88100-ce71-48a2-bf56-624f61f57c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:35.692866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb88100-ce71-48a2-bf56-624f61f57c99 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:35.743565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 881/1747] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:35 2022] POST /identity/v3/projects => generated 387 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:50:35.747404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52d254-1b5d-4f77-9fd6-9fa264fe34f6 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:35.775888 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52d254-1b5d-4f77-9fd6-9fa264fe34f6 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:35.927156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff945b9-5849-4aaa-acf4-8417e6fe5c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:35.928414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff945b9-5849-4aaa-acf4-8417e6fe5c1e admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:35.929425 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff945b9-5849-4aaa-acf4-8417e6fe5c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:35.930005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff945b9-5849-4aaa-acf4-8417e6fe5c1e admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:35.931088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff945b9-5849-4aaa-acf4-8417e6fe5c1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:35.933310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff945b9-5849-4aaa-acf4-8417e6fe5c1e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:35.981295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 867/1748] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:35 2022] POST /identity/v3/users => generated 294 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:35.986872 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb88100-ce71-48a2-bf56-624f61f57c99 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:36.006578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb88100-ce71-48a2-bf56-624f61f57c99 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:36.016569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c2f25-580c-46a3-af0d-208f63623452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:36.020842 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c2f25-580c-46a3-af0d-208f63623452 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:36.021765 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c2f25-580c-46a3-af0d-208f63623452 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:36.022449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c2f25-580c-46a3-af0d-208f63623452 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:36.024605 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c2f25-580c-46a3-af0d-208f63623452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:36.029397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c2f25-580c-46a3-af0d-208f63623452 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:36.047424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 882/1749] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:36.050329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff945b9-5849-4aaa-acf4-8417e6fe5c1e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:36.067689 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff945b9-5849-4aaa-acf4-8417e6fe5c1e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:36.075715 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588da06-403d-42ba-a7ac-14fcf06ba5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:36.078822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588da06-403d-42ba-a7ac-14fcf06ba5fe admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:36.082450 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588da06-403d-42ba-a7ac-14fcf06ba5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:36.083785 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588da06-403d-42ba-a7ac-14fcf06ba5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:36.085781 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588da06-403d-42ba-a7ac-14fcf06ba5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:36.090933 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588da06-403d-42ba-a7ac-14fcf06ba5fe admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:36.136133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 868/1750] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:50:36.138046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c2f25-580c-46a3-af0d-208f63623452 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:36.158468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c2f25-580c-46a3-af0d-208f63623452 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:36.176579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061f4b9-7bed-48e0-bbe9-2420960299bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:36.179175 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061f4b9-7bed-48e0-bbe9-2420960299bc admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:36.179619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061f4b9-7bed-48e0-bbe9-2420960299bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:36.179806 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061f4b9-7bed-48e0-bbe9-2420960299bc admin admin] PATH_INFO: `/v3/projects/53fa51e4217749cab596bc3d4a8d9998/users/e134128200db446cbf3bf073ff6a1b32/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:36.231690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061f4b9-7bed-48e0-bbe9-2420960299bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=53fa51e4217749cab596bc3d4a8d9998, user_id=e134128200db446cbf3bf073ff6a1b32, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:36.242915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061f4b9-7bed-48e0-bbe9-2420960299bc admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:36.287264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 883/1751] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:36 2022] PUT /identity/v3/projects/53fa51e4217749cab596bc3d4a8d9998/users/e134128200db446cbf3bf073ff6a1b32/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:36.300006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80d324-b999-4a66-8f2f-9bae9371b4c0 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:36.300267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80d324-b999-4a66-8f2f-9bae9371b4c0 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:36.300613 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80d324-b999-4a66-8f2f-9bae9371b4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:36.410305 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ec80d324-b999-4a66-8f2f-9bae9371b4c0 None None] MFA Rules not processed for user `e134128200db446cbf3bf073ff6a1b32`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:36.412024 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ec80d324-b999-4a66-8f2f-9bae9371b4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:36.619082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 869/1752] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:36.623397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061f4b9-7bed-48e0-bbe9-2420960299bc admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:36.786318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:36.795430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061f4b9-7bed-48e0-bbe9-2420960299bc admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:36.837958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07432ee-e41c-48a5-8c0b-5519a4b48c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:36.840141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07432ee-e41c-48a5-8c0b-5519a4b48c6c admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:36.840632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07432ee-e41c-48a5-8c0b-5519a4b48c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:36.841587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07432ee-e41c-48a5-8c0b-5519a4b48c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:36.844021 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07432ee-e41c-48a5-8c0b-5519a4b48c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:36.857349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07432ee-e41c-48a5-8c0b-5519a4b48c6c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:36.942969 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 884/1753] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:36 2022] POST /identity/v3/projects => generated 387 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:36.947650 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80d324-b999-4a66-8f2f-9bae9371b4c0 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:36.964880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80d324-b999-4a66-8f2f-9bae9371b4c0 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:37.008328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07432ee-e41c-48a5-8c0b-5519a4b48c6c admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:37.009878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a07432ee-e41c-48a5-8c0b-5519a4b48c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:37.085460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:37.107320 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c6306-1fa8-47dd-bf0b-62ad54561aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:37.109080 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c6306-1fa8-47dd-bf0b-62ad54561aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:37.109486 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c6306-1fa8-47dd-bf0b-62ad54561aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:37.109773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c6306-1fa8-47dd-bf0b-62ad54561aa1 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:37.110490 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c6306-1fa8-47dd-bf0b-62ad54561aa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:37.112747 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c6306-1fa8-47dd-bf0b-62ad54561aa1 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:37.205986 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 870/1754] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:36 2022] POST /identity/v3/users => generated 300 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:37.211289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c6306-1fa8-47dd-bf0b-62ad54561aa1 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:37.242200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c6306-1fa8-47dd-bf0b-62ad54561aa1 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:37.262427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f03c6-9d14-4c71-8fde-735d71c319d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:37.262796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f03c6-9d14-4c71-8fde-735d71c319d5 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:37.263100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f03c6-9d14-4c71-8fde-735d71c319d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:37.263341 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f03c6-9d14-4c71-8fde-735d71c319d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:37.264984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f03c6-9d14-4c71-8fde-735d71c319d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:37.266746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f03c6-9d14-4c71-8fde-735d71c319d5 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:37.291394 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07432ee-e41c-48a5-8c0b-5519a4b48c6c admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:37.314572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 871/1755] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:37.318837 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:37.319971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f03c6-9d14-4c71-8fde-735d71c319d5 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:37.320185 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:37.320185 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:37.320185 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b admin admin] PATH_INFO: `/v3/projects/f7d2f4260b1b44c7b145bd0e013f1fda` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:37.323016 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d2f4260b1b44c7b145bd0e013f1fda)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:37.329423 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:37.347624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f03c6-9d14-4c71-8fde-735d71c319d5 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:37.377386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956ef33-21cd-4874-bfcf-3dbb699e8770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:37.378092 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956ef33-21cd-4874-bfcf-3dbb699e8770 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:37.378377 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956ef33-21cd-4874-bfcf-3dbb699e8770 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:37.378647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956ef33-21cd-4874-bfcf-3dbb699e8770 admin admin] PATH_INFO: `/v3/projects/3b024e4b05894e2e83a9a521be55562d/users/56931a101e6c4ec7a0310ef31d45180b/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:37.440217 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956ef33-21cd-4874-bfcf-3dbb699e8770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b024e4b05894e2e83a9a521be55562d, user_id=56931a101e6c4ec7a0310ef31d45180b, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:37.447044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956ef33-21cd-4874-bfcf-3dbb699e8770 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:37.489991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d2f4260b1b44c7b145bd0e013f1fda', 'request_id': 'req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:37.520165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d2f4260b1b44c7b145bd0e013f1fda', 'request_id': 'req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:37.535724 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 872/1756] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:37 2022] PUT /identity/v3/projects/3b024e4b05894e2e83a9a521be55562d/users/56931a101e6c4ec7a0310ef31d45180b/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:37.539827 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956ef33-21cd-4874-bfcf-3dbb699e8770 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:37.543415 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d2f4260b1b44c7b145bd0e013f1fda', 'request_id': 'req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:37.599739 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b admin admin] The token cache is being invalidate because project f7d2f4260b1b44c7b145bd0e013f1fda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:37.601410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:37.604521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:37.611609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 885/1757] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:37 2022] DELETE /identity/v3/projects/f7d2f4260b1b44c7b145bd0e013f1fda => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:37.615069 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:37.623126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:37.777719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956ef33-21cd-4874-bfcf-3dbb699e8770 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:37.797252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58fea4c-8c39-4469-a047-58bce940326a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:37.797913 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58fea4c-8c39-4469-a047-58bce940326a admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:37.798201 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58fea4c-8c39-4469-a047-58bce940326a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:37.799413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58fea4c-8c39-4469-a047-58bce940326a admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:37.799413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58fea4c-8c39-4469-a047-58bce940326a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:37.801421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58fea4c-8c39-4469-a047-58bce940326a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:37.851582 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 873/1758] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:37.852913 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58fea4c-8c39-4469-a047-58bce940326a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:37.863027 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f58fea4c-8c39-4469-a047-58bce940326a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:37.896265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01ea0b-7c36-47c5-b4a7-e4cf44da1a3b admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:37.901748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade627a-ffbf-4af8-b144-305515dc66f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:37.902650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade627a-ffbf-4af8-b144-305515dc66f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:37.903259 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade627a-ffbf-4af8-b144-305515dc66f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:37.903732 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade627a-ffbf-4af8-b144-305515dc66f4 admin admin] PATH_INFO: `/v3/users/b5e2de8372be47af8c90f29dd5c6d9c5` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:37.915088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58fea4c-8c39-4469-a047-58bce940326a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:37.919496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade627a-ffbf-4af8-b144-305515dc66f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5e2de8372be47af8c90f29dd5c6d9c5)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:37.923158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade627a-ffbf-4af8-b144-305515dc66f4 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:37.945002 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:37.949104 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c30bad-585d-4f1a-8d39-928e1fab44b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:37.949800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c30bad-585d-4f1a-8d39-928e1fab44b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:37.950088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c30bad-585d-4f1a-8d39-928e1fab44b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:37.950366 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c30bad-585d-4f1a-8d39-928e1fab44b1 admin admin] PATH_INFO: `/v3/projects/3b024e4b05894e2e83a9a521be55562d/users/56931a101e6c4ec7a0310ef31d45180b/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:37.979952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c30bad-585d-4f1a-8d39-928e1fab44b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b024e4b05894e2e83a9a521be55562d, user_id=56931a101e6c4ec7a0310ef31d45180b, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:38.000605 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c30bad-585d-4f1a-8d39-928e1fab44b1 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:38.031841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0ade627a-ffbf-4af8-b144-305515dc66f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5e2de8372be47af8c90f29dd5c6d9c5'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:38.035927 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 874/1759] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:37 2022] PUT /identity/v3/projects/3b024e4b05894e2e83a9a521be55562d/users/56931a101e6c4ec7a0310ef31d45180b/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:38.048645 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1438bf-d66c-466e-b273-5eafc6cf296f None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:38.048942 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1438bf-d66c-466e-b273-5eafc6cf296f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:38.051962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1438bf-d66c-466e-b273-5eafc6cf296f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:38.084090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0ade627a-ffbf-4af8-b144-305515dc66f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5e2de8372be47af8c90f29dd5c6d9c5'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:38.117972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0ade627a-ffbf-4af8-b144-305515dc66f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5e2de8372be47af8c90f29dd5c6d9c5'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:38.157600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-8e1438bf-d66c-466e-b273-5eafc6cf296f None None] MFA Rules not processed for user `56931a101e6c4ec7a0310ef31d45180b`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:38.158859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8e1438bf-d66c-466e-b273-5eafc6cf296f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:38.159364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0ade627a-ffbf-4af8-b144-305515dc66f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5e2de8372be47af8c90f29dd5c6d9c5'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:38.161955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0ade627a-ffbf-4af8-b144-305515dc66f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5e2de8372be47af8c90f29dd5c6d9c5'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:38.176681 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 886/1760] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:37 2022] DELETE /identity/v3/users/b5e2de8372be47af8c90f29dd5c6d9c5 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:38.341405 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 875/1761] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:38.364913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade627a-ffbf-4af8-b144-305515dc66f4 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:38.366557 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0ade627a-ffbf-4af8-b144-305515dc66f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:38.545385 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade627a-ffbf-4af8-b144-305515dc66f4 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:38.557434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac89ec-d232-4b68-bad5-816fc2ce7f86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:38.558495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac89ec-d232-4b68-bad5-816fc2ce7f86 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:38.559468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac89ec-d232-4b68-bad5-816fc2ce7f86 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:38.560085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac89ec-d232-4b68-bad5-816fc2ce7f86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:38.830232 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac89ec-d232-4b68-bad5-816fc2ce7f86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:38.831855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac89ec-d232-4b68-bad5-816fc2ce7f86 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:38.846367 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 887/1762] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:38.996221 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1438bf-d66c-466e-b273-5eafc6cf296f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:39.001728 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8e1438bf-d66c-466e-b273-5eafc6cf296f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:39.055247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:39.172712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1438bf-d66c-466e-b273-5eafc6cf296f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:39.322517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9777cee-1811-404c-a79f-cd88764c4c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:39.323522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9777cee-1811-404c-a79f-cd88764c4c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:39.323877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9777cee-1811-404c-a79f-cd88764c4c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:39.324214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9777cee-1811-404c-a79f-cd88764c4c24 admin admin] PATH_INFO: `/v3/users/e134128200db446cbf3bf073ff6a1b32` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:39.330476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9777cee-1811-404c-a79f-cd88764c4c24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e134128200db446cbf3bf073ff6a1b32)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:39.333803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9777cee-1811-404c-a79f-cd88764c4c24 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:39.443958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b9777cee-1811-404c-a79f-cd88764c4c24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e134128200db446cbf3bf073ff6a1b32'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:39.468692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b9777cee-1811-404c-a79f-cd88764c4c24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e134128200db446cbf3bf073ff6a1b32'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:39.502420 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b9777cee-1811-404c-a79f-cd88764c4c24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e134128200db446cbf3bf073ff6a1b32'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:39.521499 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b9777cee-1811-404c-a79f-cd88764c4c24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e134128200db446cbf3bf073ff6a1b32'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:39.526230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b9777cee-1811-404c-a79f-cd88764c4c24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e134128200db446cbf3bf073ff6a1b32'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:39.549940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 876/1763] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:38 2022] DELETE /identity/v3/users/e134128200db446cbf3bf073ff6a1b32 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:39.557288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dce99-e4a4-483f-a1f9-52fde862a451 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:39.557812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dce99-e4a4-483f-a1f9-52fde862a451 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:39.557894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dce99-e4a4-483f-a1f9-52fde862a451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:39.613542 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-2c0dce99-e4a4-483f-a1f9-52fde862a451 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:39.614939 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2c0dce99-e4a4-483f-a1f9-52fde862a451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:39.756261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:39.848066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 888/1764] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:39.906322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9777cee-1811-404c-a79f-cd88764c4c24 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:39.908942 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b9777cee-1811-404c-a79f-cd88764c4c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:40.028905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:40.074163 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9777cee-1811-404c-a79f-cd88764c4c24 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:40.085344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e400f-846d-48d7-af06-26cd440d6b42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:40.085999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e400f-846d-48d7-af06-26cd440d6b42 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:40.086345 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e400f-846d-48d7-af06-26cd440d6b42 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:40.086623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e400f-846d-48d7-af06-26cd440d6b42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:40.109409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e400f-846d-48d7-af06-26cd440d6b42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:40.111001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e400f-846d-48d7-af06-26cd440d6b42 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:40.137625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 877/1765] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:40.984960 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da5ab4-6c2a-4755-b62a-f2b00d10fd84 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:40.986539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da5ab4-6c2a-4755-b62a-f2b00d10fd84 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:40.987091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da5ab4-6c2a-4755-b62a-f2b00d10fd84 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:40.989533 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 889/1766] 10.222.0.69 () {54 vars in 921 bytes} [Wed Aug 3 18:50:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:50:41.020050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9820f8-2e5e-486b-a07d-78d5364c1eb1 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:41.020657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9820f8-2e5e-486b-a07d-78d5364c1eb1 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:41.020753 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9820f8-2e5e-486b-a07d-78d5364c1eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:41.078260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-8e9820f8-2e5e-486b-a07d-78d5364c1eb1 None None] MFA Rules not processed for user `850a50cfe4394febbafb0908b08888b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:41.080529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8e9820f8-2e5e-486b-a07d-78d5364c1eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:41.190322 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da5ab4-6c2a-4755-b62a-f2b00d10fd84 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:41.192800 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c4da5ab4-6c2a-4755-b62a-f2b00d10fd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:41.237773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 878/1767] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Aug 3 18:50:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:41.239824 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da5ab4-6c2a-4755-b62a-f2b00d10fd84 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:41.253894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a8133-8a86-4e00-bffb-b0491a13b043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:41.254798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a8133-8a86-4e00-bffb-b0491a13b043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:41.255083 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a8133-8a86-4e00-bffb-b0491a13b043 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:41.255360 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a8133-8a86-4e00-bffb-b0491a13b043 admin admin] PATH_INFO: `/v3/projects/b42450eacc8f4716b764d203e27d987e` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:41.259884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a8133-8a86-4e00-bffb-b0491a13b043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b42450eacc8f4716b764d203e27d987e)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:41.260763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a8133-8a86-4e00-bffb-b0491a13b043 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:41.284548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9820f8-2e5e-486b-a07d-78d5364c1eb1 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:41.286074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8e9820f8-2e5e-486b-a07d-78d5364c1eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:41.316910 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9820f8-2e5e-486b-a07d-78d5364c1eb1 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:41.363224 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbf6d7-6604-48de-b5a9-8004ac0198fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:41.363886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbf6d7-6604-48de-b5a9-8004ac0198fd service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:41.364174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbf6d7-6604-48de-b5a9-8004ac0198fd service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:41.364483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbf6d7-6604-48de-b5a9-8004ac0198fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:41.401777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dbf6d7-6604-48de-b5a9-8004ac0198fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:41.406668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dbf6d7-6604-48de-b5a9-8004ac0198fd service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:41.462456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-150a8133-8a86-4e00-bffb-b0491a13b043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b42450eacc8f4716b764d203e27d987e', 'request_id': 'req-150a8133-8a86-4e00-bffb-b0491a13b043'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:41.464657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 879/1768] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:41.478958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-150a8133-8a86-4e00-bffb-b0491a13b043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b42450eacc8f4716b764d203e27d987e', 'request_id': 'req-150a8133-8a86-4e00-bffb-b0491a13b043'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:41.502093 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-150a8133-8a86-4e00-bffb-b0491a13b043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b42450eacc8f4716b764d203e27d987e', 'request_id': 'req-150a8133-8a86-4e00-bffb-b0491a13b043'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:41.526142 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-150a8133-8a86-4e00-bffb-b0491a13b043 admin admin] The token cache is being invalidate because project b42450eacc8f4716b764d203e27d987e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:41.526439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-150a8133-8a86-4e00-bffb-b0491a13b043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:41.527190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-150a8133-8a86-4e00-bffb-b0491a13b043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:41.529452 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 890/1769] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:41 2022] DELETE /identity/v3/projects/b42450eacc8f4716b764d203e27d987e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:50:41.534444 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbdf02-66cd-49e0-af59-de8d523ed704 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:41.534967 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbdf02-66cd-49e0-af59-de8d523ed704 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:41.535392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbdf02-66cd-49e0-af59-de8d523ed704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:41.560390 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4489cf-0570-402f-ac2b-34895398ccb0 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:41.562334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4489cf-0570-402f-ac2b-34895398ccb0 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:41.562610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4489cf-0570-402f-ac2b-34895398ccb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:41.597789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-e5dbdf02-66cd-49e0-af59-de8d523ed704 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:41.614795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e5dbdf02-66cd-49e0-af59-de8d523ed704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:41.678098 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-7c4489cf-0570-402f-ac2b-34895398ccb0 None None] MFA Rules not processed for user `c84338b02b474eaa9f02288d679da181`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:41.684467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7c4489cf-0570-402f-ac2b-34895398ccb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:41.880885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 880/1770] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:41.883099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2c6e2-7cd7-460d-b067-ce954992429f None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:41.883548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2c6e2-7cd7-460d-b067-ce954992429f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:41.883980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2c6e2-7cd7-460d-b067-ce954992429f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:41.949538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 891/1771] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:41 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:41.949538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c71bfb-7ed7-41ba-a165-841f58a6d7d6 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:41.949811 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-58d2c6e2-7cd7-460d-b067-ce954992429f None None] MFA Rules not processed for user `13ab977f852a41e898af1ddfb67bad69`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:41.949905 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c71bfb-7ed7-41ba-a165-841f58a6d7d6 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:41.949905 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c71bfb-7ed7-41ba-a165-841f58a6d7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:41.950286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-58d2c6e2-7cd7-460d-b067-ce954992429f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:41.991327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-74c71bfb-7ed7-41ba-a165-841f58a6d7d6 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:41.994523 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-74c71bfb-7ed7-41ba-a165-841f58a6d7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:42.014959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 892/1772] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:42.016587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c71bfb-7ed7-41ba-a165-841f58a6d7d6 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:42.039169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c71bfb-7ed7-41ba-a165-841f58a6d7d6 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:42.051892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1ecb1-3d85-47ca-94a7-64d6a898cc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:42.052580 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1ecb1-3d85-47ca-94a7-64d6a898cc92 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:42.052914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1ecb1-3d85-47ca-94a7-64d6a898cc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:42.053113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1ecb1-3d85-47ca-94a7-64d6a898cc92 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:42.054335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1ecb1-3d85-47ca-94a7-64d6a898cc92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:42.054634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1ecb1-3d85-47ca-94a7-64d6a898cc92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:42.062286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1ecb1-3d85-47ca-94a7-64d6a898cc92 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:42.091465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 881/1773] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:42.093973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 893/1774] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:42.100533 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2c6e2-7cd7-460d-b067-ce954992429f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:42.119416 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2c6e2-7cd7-460d-b067-ce954992429f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:42.131631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad53ed-19e5-469e-8bae-fa426916f291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:42.132316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad53ed-19e5-469e-8bae-fa426916f291 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:42.132620 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad53ed-19e5-469e-8bae-fa426916f291 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:42.132894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad53ed-19e5-469e-8bae-fa426916f291 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:42.133580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad53ed-19e5-469e-8bae-fa426916f291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:42.137309 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad53ed-19e5-469e-8bae-fa426916f291 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:42.152939 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:42.244025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 882/1775] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:42 2022] POST /identity/v3/projects => generated 365 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:42.247255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1ecb1-3d85-47ca-94a7-64d6a898cc92 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:42.265865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1ecb1-3d85-47ca-94a7-64d6a898cc92 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:42.306286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:42.353225 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293e414-793b-49dd-bfa7-04e1c3ad93c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:42.354335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293e414-793b-49dd-bfa7-04e1c3ad93c4 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:42.354769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293e414-793b-49dd-bfa7-04e1c3ad93c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:42.355187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293e414-793b-49dd-bfa7-04e1c3ad93c4 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:42.355987 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293e414-793b-49dd-bfa7-04e1c3ad93c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:42.358959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293e414-793b-49dd-bfa7-04e1c3ad93c4 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:42.365995 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad53ed-19e5-469e-8bae-fa426916f291 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:42.368079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-61ad53ed-19e5-469e-8bae-fa426916f291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:42.391029 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad53ed-19e5-469e-8bae-fa426916f291 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:42.401350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98928f85-1a78-4d1c-b9fe-e4b3386dd016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:42.402021 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98928f85-1a78-4d1c-b9fe-e4b3386dd016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:42.402407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98928f85-1a78-4d1c-b9fe-e4b3386dd016 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:42.402900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98928f85-1a78-4d1c-b9fe-e4b3386dd016 admin admin] PATH_INFO: `/v3/projects/53fa51e4217749cab596bc3d4a8d9998` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:42.404088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98928f85-1a78-4d1c-b9fe-e4b3386dd016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53fa51e4217749cab596bc3d4a8d9998)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:42.416459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98928f85-1a78-4d1c-b9fe-e4b3386dd016 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:42.416957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 894/1776] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:42 2022] POST /identity/v3/users => generated 283 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:42.419780 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293e414-793b-49dd-bfa7-04e1c3ad93c4 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:42.439245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293e414-793b-49dd-bfa7-04e1c3ad93c4 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:42.453733 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5a91a-ce96-44bb-9086-932ff2304097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:42.454439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5a91a-ce96-44bb-9086-932ff2304097 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:42.454736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5a91a-ce96-44bb-9086-932ff2304097 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:42.454992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5a91a-ce96-44bb-9086-932ff2304097 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:42.455697 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5a91a-ce96-44bb-9086-932ff2304097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:42.458725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5a91a-ce96-44bb-9086-932ff2304097 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:42.482973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 895/1777] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:42.495390 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5a91a-ce96-44bb-9086-932ff2304097 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:42.523579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5a91a-ce96-44bb-9086-932ff2304097 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:42.531382 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-98928f85-1a78-4d1c-b9fe-e4b3386dd016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53fa51e4217749cab596bc3d4a8d9998', 'request_id': 'req-98928f85-1a78-4d1c-b9fe-e4b3386dd016'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:42.547748 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-98928f85-1a78-4d1c-b9fe-e4b3386dd016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53fa51e4217749cab596bc3d4a8d9998', 'request_id': 'req-98928f85-1a78-4d1c-b9fe-e4b3386dd016'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:42.561701 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-98928f85-1a78-4d1c-b9fe-e4b3386dd016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53fa51e4217749cab596bc3d4a8d9998', 'request_id': 'req-98928f85-1a78-4d1c-b9fe-e4b3386dd016'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:42.628342 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-98928f85-1a78-4d1c-b9fe-e4b3386dd016 admin admin] The token cache is being invalidate because project 53fa51e4217749cab596bc3d4a8d9998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:42.652365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-98928f85-1a78-4d1c-b9fe-e4b3386dd016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:42.654066 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-98928f85-1a78-4d1c-b9fe-e4b3386dd016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:42.658734 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 883/1778] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:42 2022] DELETE /identity/v3/projects/53fa51e4217749cab596bc3d4a8d9998 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:50:42.662980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98928f85-1a78-4d1c-b9fe-e4b3386dd016 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:42.665926 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-98928f85-1a78-4d1c-b9fe-e4b3386dd016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:42.729424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7dbd82-f16c-4e12-a0d6-c2fc5198bdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:42.730836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dbd82-f16c-4e12-a0d6-c2fc5198bdb6 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:42.731193 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dbd82-f16c-4e12-a0d6-c2fc5198bdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:42.732023 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dbd82-f16c-4e12-a0d6-c2fc5198bdb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:42.736982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7dbd82-f16c-4e12-a0d6-c2fc5198bdb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:42.740931 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7dbd82-f16c-4e12-a0d6-c2fc5198bdb6 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:42.771246 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 896/1779] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:42.781526 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7dbd82-f16c-4e12-a0d6-c2fc5198bdb6 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:42.784930 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-cc7dbd82-f16c-4e12-a0d6-c2fc5198bdb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:42.810841 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98928f85-1a78-4d1c-b9fe-e4b3386dd016 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:42.811935 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7dbd82-f16c-4e12-a0d6-c2fc5198bdb6 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:42.818640 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42471878-8960-4d58-8203-b2c0dfa3ace6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:42.819811 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42471878-8960-4d58-8203-b2c0dfa3ace6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:42.820243 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42471878-8960-4d58-8203-b2c0dfa3ace6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:42.820594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a1525-307b-486e-8f19-6678111f15c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:42.820868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42471878-8960-4d58-8203-b2c0dfa3ace6 admin admin] PATH_INFO: `/v3/users/56931a101e6c4ec7a0310ef31d45180b` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:42.822233 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42471878-8960-4d58-8203-b2c0dfa3ace6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56931a101e6c4ec7a0310ef31d45180b)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:42.822583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a1525-307b-486e-8f19-6678111f15c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:42.823142 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a1525-307b-486e-8f19-6678111f15c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:42.824817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42471878-8960-4d58-8203-b2c0dfa3ace6 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:42.827192 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a1525-307b-486e-8f19-6678111f15c0 admin admin] PATH_INFO: `/v3/projects/99c3acb259d64e7ea2648cee0b2cd353/users/3ee7c9eab128461782f95adf5cda294f/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:42.895217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a1525-307b-486e-8f19-6678111f15c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c3acb259d64e7ea2648cee0b2cd353, user_id=3ee7c9eab128461782f95adf5cda294f, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:42.912574 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a1525-307b-486e-8f19-6678111f15c0 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:42.920145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-42471878-8960-4d58-8203-b2c0dfa3ace6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56931a101e6c4ec7a0310ef31d45180b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:42.947792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-42471878-8960-4d58-8203-b2c0dfa3ace6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56931a101e6c4ec7a0310ef31d45180b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:42.961376 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 897/1780] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:42 2022] PUT /identity/v3/projects/99c3acb259d64e7ea2648cee0b2cd353/users/3ee7c9eab128461782f95adf5cda294f/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:42.974865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f13b9-de55-433c-a8db-0443beee07db None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:42.975604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f13b9-de55-433c-a8db-0443beee07db None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:42.975808 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f13b9-de55-433c-a8db-0443beee07db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:42.980779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:42.986808 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-42471878-8960-4d58-8203-b2c0dfa3ace6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56931a101e6c4ec7a0310ef31d45180b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:43.006511 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-42471878-8960-4d58-8203-b2c0dfa3ace6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56931a101e6c4ec7a0310ef31d45180b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:43.007347 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-42471878-8960-4d58-8203-b2c0dfa3ace6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56931a101e6c4ec7a0310ef31d45180b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:43.017692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 884/1781] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:42 2022] DELETE /identity/v3/users/56931a101e6c4ec7a0310ef31d45180b => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:50:43.025163 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-b41f13b9-de55-433c-a8db-0443beee07db None None] MFA Rules not processed for user `3ee7c9eab128461782f95adf5cda294f`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:43.026299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b41f13b9-de55-433c-a8db-0443beee07db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:43.208818 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 898/1782] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:50:43.211195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42471878-8960-4d58-8203-b2c0dfa3ace6 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:43.213404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-42471878-8960-4d58-8203-b2c0dfa3ace6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:43.315721 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42471878-8960-4d58-8203-b2c0dfa3ace6 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:43.324646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064960f-f1ff-4f8d-b6d5-6180c89e6d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:43.325767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064960f-f1ff-4f8d-b6d5-6180c89e6d1b admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:43.326272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064960f-f1ff-4f8d-b6d5-6180c89e6d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:43.327884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064960f-f1ff-4f8d-b6d5-6180c89e6d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:43.329089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064960f-f1ff-4f8d-b6d5-6180c89e6d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:43.331941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064960f-f1ff-4f8d-b6d5-6180c89e6d1b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:43.374021 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 885/1783] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:43 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:50:43.378645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f13b9-de55-433c-a8db-0443beee07db None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:43.396395 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f13b9-de55-433c-a8db-0443beee07db None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:43.410559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:43.430826 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b96527-0eae-4406-a6c6-8d3cc015e64b None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:43.431549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b96527-0eae-4406-a6c6-8d3cc015e64b None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:43.432060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b96527-0eae-4406-a6c6-8d3cc015e64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:43.481136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-18b96527-0eae-4406-a6c6-8d3cc015e64b None None] MFA Rules not processed for user `1aa1d56d9b154e32aa1c136b577d49f4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:43.483981 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-18b96527-0eae-4406-a6c6-8d3cc015e64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:43.541837 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1ba4e-55a0-4de4-8102-e00539bc7b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:43.542512 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1ba4e-55a0-4de4-8102-e00539bc7b61 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:43.542797 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1ba4e-55a0-4de4-8102-e00539bc7b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:43.543792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1ba4e-55a0-4de4-8102-e00539bc7b61 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:43.545678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1ba4e-55a0-4de4-8102-e00539bc7b61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:43.547677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1ba4e-55a0-4de4-8102-e00539bc7b61 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:43.594824 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 899/1784] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:43 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:50:43.598805 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1ba4e-55a0-4de4-8102-e00539bc7b61 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:43.617926 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1ba4e-55a0-4de4-8102-e00539bc7b61 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:43.633014 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa10239-bb25-4ef5-99fa-8fed535cf2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:43.634593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa10239-bb25-4ef5-99fa-8fed535cf2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:43.635141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa10239-bb25-4ef5-99fa-8fed535cf2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:43.635734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa10239-bb25-4ef5-99fa-8fed535cf2a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:43.636100 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa10239-bb25-4ef5-99fa-8fed535cf2a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:43.637782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa10239-bb25-4ef5-99fa-8fed535cf2a9 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:43.648699 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 886/1785] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:43.664080 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 900/1786] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:43.672751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b96527-0eae-4406-a6c6-8d3cc015e64b None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:43.695143 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b96527-0eae-4406-a6c6-8d3cc015e64b None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:43.699669 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa10239-bb25-4ef5-99fa-8fed535cf2a9 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:43.705073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-daa10239-bb25-4ef5-99fa-8fed535cf2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:43.715293 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422efc57-dd6f-4e7a-a4cc-521644ebe6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:43.716569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422efc57-dd6f-4e7a-a4cc-521644ebe6dd admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:43.717069 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422efc57-dd6f-4e7a-a4cc-521644ebe6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:43.718148 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422efc57-dd6f-4e7a-a4cc-521644ebe6dd admin admin] PATH_INFO: `/v3/projects/a3b2f2b097c94b84ab2733501432ec32/users/0e203b8f85754bcbb878b09a2501b95b/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:43.729900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa10239-bb25-4ef5-99fa-8fed535cf2a9 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:43.743334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1abaf-1e3e-4901-a7c2-338ea581716b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:43.744352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1abaf-1e3e-4901-a7c2-338ea581716b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:43.745047 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1abaf-1e3e-4901-a7c2-338ea581716b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:43.747090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1abaf-1e3e-4901-a7c2-338ea581716b admin admin] PATH_INFO: `/v3/projects/3b024e4b05894e2e83a9a521be55562d` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:43.750696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422efc57-dd6f-4e7a-a4cc-521644ebe6dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3b2f2b097c94b84ab2733501432ec32, user_id=0e203b8f85754bcbb878b09a2501b95b, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:43.751612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f1abaf-1e3e-4901-a7c2-338ea581716b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b024e4b05894e2e83a9a521be55562d)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:43.753755 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f1abaf-1e3e-4901-a7c2-338ea581716b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:43.753981 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422efc57-dd6f-4e7a-a4cc-521644ebe6dd admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:43.776006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 887/1787] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:43 2022] PUT /identity/v3/projects/a3b2f2b097c94b84ab2733501432ec32/users/0e203b8f85754bcbb878b09a2501b95b/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:43.780566 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422efc57-dd6f-4e7a-a4cc-521644ebe6dd admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:43.844127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-f0f1abaf-1e3e-4901-a7c2-338ea581716b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b024e4b05894e2e83a9a521be55562d', 'request_id': 'req-f0f1abaf-1e3e-4901-a7c2-338ea581716b'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:43.853141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-f0f1abaf-1e3e-4901-a7c2-338ea581716b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b024e4b05894e2e83a9a521be55562d', 'request_id': 'req-f0f1abaf-1e3e-4901-a7c2-338ea581716b'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:43.854025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:43.867609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-f0f1abaf-1e3e-4901-a7c2-338ea581716b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b024e4b05894e2e83a9a521be55562d', 'request_id': 'req-f0f1abaf-1e3e-4901-a7c2-338ea581716b'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:43.869881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422efc57-dd6f-4e7a-a4cc-521644ebe6dd admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:43.886538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-f0f1abaf-1e3e-4901-a7c2-338ea581716b admin admin] The token cache is being invalidate because project 3b024e4b05894e2e83a9a521be55562d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:43.887166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-f0f1abaf-1e3e-4901-a7c2-338ea581716b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:43.888534 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-f0f1abaf-1e3e-4901-a7c2-338ea581716b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:43.891632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 901/1788] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:43 2022] DELETE /identity/v3/projects/3b024e4b05894e2e83a9a521be55562d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:50:43.896806 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf9133-8c57-4952-8560-56eb5337263a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:43.897163 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf9133-8c57-4952-8560-56eb5337263a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:43.897536 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf9133-8c57-4952-8560-56eb5337263a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:43.921615 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-05cf9133-8c57-4952-8560-56eb5337263a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:43.922923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-05cf9133-8c57-4952-8560-56eb5337263a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:43.959339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67c1a6-053f-4785-b3db-a93e8d14604f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:43.960200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67c1a6-053f-4785-b3db-a93e8d14604f admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:43.960512 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67c1a6-053f-4785-b3db-a93e8d14604f admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:43.960795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67c1a6-053f-4785-b3db-a93e8d14604f admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:43.961541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67c1a6-053f-4785-b3db-a93e8d14604f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:43.963466 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67c1a6-053f-4785-b3db-a93e8d14604f admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:43.976909 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 888/1789] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:50:43.981189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67c1a6-053f-4785-b3db-a93e8d14604f admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:43.983854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1a67c1a6-053f-4785-b3db-a93e8d14604f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:44.000439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67c1a6-053f-4785-b3db-a93e8d14604f admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:44.004873 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 902/1790] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:44.010103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c1c7d-f792-4eac-81f4-da03ba9d6cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:44.010956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c1c7d-f792-4eac-81f4-da03ba9d6cff admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:44.011227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c1c7d-f792-4eac-81f4-da03ba9d6cff admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:44.012639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c1c7d-f792-4eac-81f4-da03ba9d6cff admin admin] PATH_INFO: `/v3/projects/a3b2f2b097c94b84ab2733501432ec32/users/0e203b8f85754bcbb878b09a2501b95b/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:44.015896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c1c7d-f792-4eac-81f4-da03ba9d6cff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3b2f2b097c94b84ab2733501432ec32, user_id=0e203b8f85754bcbb878b09a2501b95b, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:44.019460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c1c7d-f792-4eac-81f4-da03ba9d6cff admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:44.024191 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113ea7b-d356-457b-bab5-6d950cd2e6e4 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:44.025371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113ea7b-d356-457b-bab5-6d950cd2e6e4 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:44.031631 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113ea7b-d356-457b-bab5-6d950cd2e6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:44.036111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 889/1791] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:43 2022] PUT /identity/v3/projects/a3b2f2b097c94b84ab2733501432ec32/users/0e203b8f85754bcbb878b09a2501b95b/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:50:44.047484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf40c7-8a15-4350-922f-0dc20f020ca4 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:44.047941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf40c7-8a15-4350-922f-0dc20f020ca4 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:44.048054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf40c7-8a15-4350-922f-0dc20f020ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:44.077383 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-7113ea7b-d356-457b-bab5-6d950cd2e6e4 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:44.078645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7113ea7b-d356-457b-bab5-6d950cd2e6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:44.103174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-30cf40c7-8a15-4350-922f-0dc20f020ca4 None None] MFA Rules not processed for user `0e203b8f85754bcbb878b09a2501b95b`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:44.105639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-30cf40c7-8a15-4350-922f-0dc20f020ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:44.111361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:44.277809 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 890/1792] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:50:44.287612 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf40c7-8a15-4350-922f-0dc20f020ca4 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:44.289434 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-30cf40c7-8a15-4350-922f-0dc20f020ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:44.311652 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 903/1793] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:44.313112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113ea7b-d356-457b-bab5-6d950cd2e6e4 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:44.324647 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113ea7b-d356-457b-bab5-6d950cd2e6e4 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:44.333612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0815dbd-bfb0-4f26-80c0-32098443eb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:44.338841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0815dbd-bfb0-4f26-80c0-32098443eb8e admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:44.339138 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0815dbd-bfb0-4f26-80c0-32098443eb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:44.339422 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0815dbd-bfb0-4f26-80c0-32098443eb8e admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:44.340735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0815dbd-bfb0-4f26-80c0-32098443eb8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:44.341052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0815dbd-bfb0-4f26-80c0-32098443eb8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:44.343782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0815dbd-bfb0-4f26-80c0-32098443eb8e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:44.368984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 904/1794] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:44.379777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0815dbd-bfb0-4f26-80c0-32098443eb8e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:44.415512 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0815dbd-bfb0-4f26-80c0-32098443eb8e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:44.426950 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fec6b3-ee94-46ac-a85e-3d6f3d56911f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:44.427919 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fec6b3-ee94-46ac-a85e-3d6f3d56911f admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:44.428214 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fec6b3-ee94-46ac-a85e-3d6f3d56911f admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:44.428521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fec6b3-ee94-46ac-a85e-3d6f3d56911f admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:44.429207 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fec6b3-ee94-46ac-a85e-3d6f3d56911f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:44.433439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf40c7-8a15-4350-922f-0dc20f020ca4 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:44.435537 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fec6b3-ee94-46ac-a85e-3d6f3d56911f admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:44.438630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00efc2b-60f5-41fc-b8f3-43e3f14e615e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:44.439875 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00efc2b-60f5-41fc-b8f3-43e3f14e615e service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:44.440234 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00efc2b-60f5-41fc-b8f3-43e3f14e615e service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:44.440630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00efc2b-60f5-41fc-b8f3-43e3f14e615e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:44.468883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00efc2b-60f5-41fc-b8f3-43e3f14e615e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:44.471009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00efc2b-60f5-41fc-b8f3-43e3f14e615e service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:44.490542 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 891/1795] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:50:44.505493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 905/1796] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:44 2022] POST /identity/v3/projects => generated 393 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:44.509496 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00efc2b-60f5-41fc-b8f3-43e3f14e615e service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:44.526372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00efc2b-60f5-41fc-b8f3-43e3f14e615e service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:44.543906 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:44.594184 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed830a-7a42-47e7-8dab-55f87d0dd724 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:44.594622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed830a-7a42-47e7-8dab-55f87d0dd724 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:44.595090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed830a-7a42-47e7-8dab-55f87d0dd724 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:44.599635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 906/1797] 10.222.0.69 () {54 vars in 921 bytes} [Wed Aug 3 18:50:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:50:44.622979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9d7c8-0516-4403-823f-1728948b0575 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:44.622979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9d7c8-0516-4403-823f-1728948b0575 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:44.622979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9d7c8-0516-4403-823f-1728948b0575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:44.698604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-82d9d7c8-0516-4403-823f-1728948b0575 None None] MFA Rules not processed for user `fea93ceb4cbc49c2b4b280a4ddc257b7`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:44.700388 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe301a-ad38-419f-8d98-66d6edaf92ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:44.701266 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe301a-ad38-419f-8d98-66d6edaf92ce admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:44.701511 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-82d9d7c8-0516-4403-823f-1728948b0575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:44.702646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe301a-ad38-419f-8d98-66d6edaf92ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:44.704215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe301a-ad38-419f-8d98-66d6edaf92ce admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:44.705159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe301a-ad38-419f-8d98-66d6edaf92ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:44.719557 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe301a-ad38-419f-8d98-66d6edaf92ce admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:44.788085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 892/1798] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:44 2022] POST /identity/v3/users => generated 297 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:44.788743 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe301a-ad38-419f-8d98-66d6edaf92ce admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:44.928015 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe301a-ad38-419f-8d98-66d6edaf92ce admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:44.945633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375decdd-514e-4942-80ac-f15b35be12fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:44.946967 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375decdd-514e-4942-80ac-f15b35be12fa service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:44.947231 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375decdd-514e-4942-80ac-f15b35be12fa service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:44.947714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375decdd-514e-4942-80ac-f15b35be12fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:44.951016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-375decdd-514e-4942-80ac-f15b35be12fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:45.000427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 907/1799] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Aug 3 18:50:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:50:45.001576 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9d7c8-0516-4403-823f-1728948b0575 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:45.043324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9d7c8-0516-4403-823f-1728948b0575 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:45.056148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fb1a5-7322-4cf5-9dc4-da6e5cd6d833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:45.059512 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fb1a5-7322-4cf5-9dc4-da6e5cd6d833 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:45.059994 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fb1a5-7322-4cf5-9dc4-da6e5cd6d833 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:45.062019 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fb1a5-7322-4cf5-9dc4-da6e5cd6d833 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:45.064653 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fb1a5-7322-4cf5-9dc4-da6e5cd6d833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:45.065725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375decdd-514e-4942-80ac-f15b35be12fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:45.067355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375decdd-514e-4942-80ac-f15b35be12fa service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:45.068226 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fb1a5-7322-4cf5-9dc4-da6e5cd6d833 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:45.090645 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 893/1800] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:45.091875 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375decdd-514e-4942-80ac-f15b35be12fa service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:45.097253 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-375decdd-514e-4942-80ac-f15b35be12fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:45.111669 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 908/1801] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:45.115834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fb1a5-7322-4cf5-9dc4-da6e5cd6d833 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:45.133646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375decdd-514e-4942-80ac-f15b35be12fa service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:45.150911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52676c-7d84-41de-99b7-d2794d0aad79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:45.151567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52676c-7d84-41de-99b7-d2794d0aad79 service placement] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:45.152133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52676c-7d84-41de-99b7-d2794d0aad79 service placement] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:45.152500 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52676c-7d84-41de-99b7-d2794d0aad79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:45.153503 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fb1a5-7322-4cf5-9dc4-da6e5cd6d833 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:45.168381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52676c-7d84-41de-99b7-d2794d0aad79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:45.171219 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52676c-7d84-41de-99b7-d2794d0aad79 service placement] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:45.175519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7230e-f1d4-4eba-a2ec-a1c3e7e52f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:45.176458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7230e-f1d4-4eba-a2ec-a1c3e7e52f4c admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:45.177358 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7230e-f1d4-4eba-a2ec-a1c3e7e52f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:45.177822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7230e-f1d4-4eba-a2ec-a1c3e7e52f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:45.178718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7230e-f1d4-4eba-a2ec-a1c3e7e52f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:45.180739 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7230e-f1d4-4eba-a2ec-a1c3e7e52f4c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:45.189362 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 894/1802] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:45.239124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 909/1803] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:50:45.243688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52676c-7d84-41de-99b7-d2794d0aad79 service placement] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:45.263072 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52676c-7d84-41de-99b7-d2794d0aad79 service placement] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:45.276363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843cc3d1-b934-4444-b5c8-e08577efbfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:45.278014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cc3d1-b934-4444-b5c8-e08577efbfc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:45.278666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cc3d1-b934-4444-b5c8-e08577efbfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:45.280671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cc3d1-b934-4444-b5c8-e08577efbfc5 admin admin] PATH_INFO: `/v3/projects/09b727efe3a0433aa7f53735a9bf9b04/users/552d619912cd45ccae9b9a6068b37e02/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:45.344653 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843cc3d1-b934-4444-b5c8-e08577efbfc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b727efe3a0433aa7f53735a9bf9b04, user_id=552d619912cd45ccae9b9a6068b37e02, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:45.349586 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843cc3d1-b934-4444-b5c8-e08577efbfc5 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:45.379039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 895/1804] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:45 2022] PUT /identity/v3/projects/09b727efe3a0433aa7f53735a9bf9b04/users/552d619912cd45ccae9b9a6068b37e02/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:45.392881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3dff7-ce57-4e92-ab6c-d92349d22bea None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:45.393226 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3dff7-ce57-4e92-ab6c-d92349d22bea None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:45.393478 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3dff7-ce57-4e92-ab6c-d92349d22bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:45.453713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-dcd3dff7-ce57-4e92-ab6c-d92349d22bea None None] MFA Rules not processed for user `552d619912cd45ccae9b9a6068b37e02`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:45.455501 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-dcd3dff7-ce57-4e92-ab6c-d92349d22bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:45.528940 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:45.575795 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 910/1805] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:45.577972 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843cc3d1-b934-4444-b5c8-e08577efbfc5 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:45.607342 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:45.678452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843cc3d1-b934-4444-b5c8-e08577efbfc5 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:45.683310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02206358-3bc9-44e4-b8bf-4a4248ba2418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:45.684011 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02206358-3bc9-44e4-b8bf-4a4248ba2418 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:45.684906 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02206358-3bc9-44e4-b8bf-4a4248ba2418 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:45.684906 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02206358-3bc9-44e4-b8bf-4a4248ba2418 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:45.685451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02206358-3bc9-44e4-b8bf-4a4248ba2418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:45.688075 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02206358-3bc9-44e4-b8bf-4a4248ba2418 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:45.728070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 896/1806] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:45 2022] POST /identity/v3/projects => generated 393 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:45.732275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3dff7-ce57-4e92-ab6c-d92349d22bea None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:45.745822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3dff7-ce57-4e92-ab6c-d92349d22bea None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:45.996090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbefef8-e014-4bba-b77e-ddfd2aee61bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:45.997617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbefef8-e014-4bba-b77e-ddfd2aee61bb admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:45.999103 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbefef8-e014-4bba-b77e-ddfd2aee61bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:45.999983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbefef8-e014-4bba-b77e-ddfd2aee61bb admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:46.001853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbefef8-e014-4bba-b77e-ddfd2aee61bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:46.005173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbefef8-e014-4bba-b77e-ddfd2aee61bb admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:46.049183 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02206358-3bc9-44e4-b8bf-4a4248ba2418 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:46.052064 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-02206358-3bc9-44e4-b8bf-4a4248ba2418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:46.058481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 911/1807] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:45 2022] POST /identity/v3/users => generated 303 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:46.058995 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbefef8-e014-4bba-b77e-ddfd2aee61bb admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:46.086519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbefef8-e014-4bba-b77e-ddfd2aee61bb admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:46.099184 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669e494-1c5c-4d4d-9d04-65d1e6801799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:46.102260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669e494-1c5c-4d4d-9d04-65d1e6801799 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:46.103521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669e494-1c5c-4d4d-9d04-65d1e6801799 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:46.103677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669e494-1c5c-4d4d-9d04-65d1e6801799 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:46.106531 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669e494-1c5c-4d4d-9d04-65d1e6801799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:46.110369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669e494-1c5c-4d4d-9d04-65d1e6801799 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:46.116161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02206358-3bc9-44e4-b8bf-4a4248ba2418 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:46.133905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8d81f-e12b-4a4b-ad47-53516371bc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:46.134970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8d81f-e12b-4a4b-ad47-53516371bc6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:46.135325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8d81f-e12b-4a4b-ad47-53516371bc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:46.135672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8d81f-e12b-4a4b-ad47-53516371bc6d admin admin] PATH_INFO: `/v3/users/246430adfe1b4144a52c3aede7307353` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:46.137291 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8d81f-e12b-4a4b-ad47-53516371bc6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=246430adfe1b4144a52c3aede7307353)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:46.142269 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8d81f-e12b-4a4b-ad47-53516371bc6d admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:46.146240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 912/1808] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:46.152815 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669e494-1c5c-4d4d-9d04-65d1e6801799 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:46.182203 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669e494-1c5c-4d4d-9d04-65d1e6801799 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:46.193189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b8b97-5f5c-4421-a439-09c2d23b8670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:46.194310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b8b97-5f5c-4421-a439-09c2d23b8670 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:46.197529 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b8b97-5f5c-4421-a439-09c2d23b8670 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:46.198098 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b8b97-5f5c-4421-a439-09c2d23b8670 admin admin] PATH_INFO: `/v3/projects/dce9de7e98844951b9b4b420512540db/users/878c2c4db7724196a617303577925c6b/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:46.232361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b8b97-5f5c-4421-a439-09c2d23b8670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce9de7e98844951b9b4b420512540db, user_id=878c2c4db7724196a617303577925c6b, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:46.237696 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b8b97-5f5c-4421-a439-09c2d23b8670 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:46.242218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-27d8d81f-e12b-4a4b-ad47-53516371bc6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '246430adfe1b4144a52c3aede7307353'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:46.289217 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-27d8d81f-e12b-4a4b-ad47-53516371bc6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '246430adfe1b4144a52c3aede7307353'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:46.300389 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 913/1809] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:46 2022] PUT /identity/v3/projects/dce9de7e98844951b9b4b420512540db/users/878c2c4db7724196a617303577925c6b/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:46.304357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b8b97-5f5c-4421-a439-09c2d23b8670 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:46.318521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-27d8d81f-e12b-4a4b-ad47-53516371bc6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '246430adfe1b4144a52c3aede7307353'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:46.332071 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-27d8d81f-e12b-4a4b-ad47-53516371bc6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '246430adfe1b4144a52c3aede7307353'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:46.332969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-27d8d81f-e12b-4a4b-ad47-53516371bc6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '246430adfe1b4144a52c3aede7307353'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:46.373220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 897/1810] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:46 2022] DELETE /identity/v3/users/246430adfe1b4144a52c3aede7307353 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:46.451229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b8b97-5f5c-4421-a439-09c2d23b8670 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:46.559184 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:46.613445 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e213ef-8171-46b7-8597-5f231e735945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:46.614815 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e213ef-8171-46b7-8597-5f231e735945 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:46.615894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e213ef-8171-46b7-8597-5f231e735945 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:46.616391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e213ef-8171-46b7-8597-5f231e735945 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:46.617500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e213ef-8171-46b7-8597-5f231e735945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:46.619982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e213ef-8171-46b7-8597-5f231e735945 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:46.651066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 914/1811] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:50:46.655333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8d81f-e12b-4a4b-ad47-53516371bc6d admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:46.657464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-27d8d81f-e12b-4a4b-ad47-53516371bc6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:46.690754 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8d81f-e12b-4a4b-ad47-53516371bc6d admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:46.701038 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f52b0-7663-4e2a-b940-79742bd20e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:46.702737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f52b0-7663-4e2a-b940-79742bd20e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:46.703963 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f52b0-7663-4e2a-b940-79742bd20e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:46.704352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f52b0-7663-4e2a-b940-79742bd20e17 admin admin] PATH_INFO: `/v3/projects/dce9de7e98844951b9b4b420512540db/users/878c2c4db7724196a617303577925c6b/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:46.713632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f52b0-7663-4e2a-b940-79742bd20e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce9de7e98844951b9b4b420512540db, user_id=878c2c4db7724196a617303577925c6b, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:46.722818 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f52b0-7663-4e2a-b940-79742bd20e17 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:46.757676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 898/1812] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:46 2022] PUT /identity/v3/projects/dce9de7e98844951b9b4b420512540db/users/878c2c4db7724196a617303577925c6b/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:46.773060 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392969a-ec4b-480b-90bc-cf0a24502b3f None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:46.773531 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392969a-ec4b-480b-90bc-cf0a24502b3f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:46.774119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392969a-ec4b-480b-90bc-cf0a24502b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:46.815959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-1392969a-ec4b-480b-90bc-cf0a24502b3f None None] MFA Rules not processed for user `878c2c4db7724196a617303577925c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:46.817431 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1392969a-ec4b-480b-90bc-cf0a24502b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:46.912692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f52b0-7663-4e2a-b940-79742bd20e17 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:46.914808 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-798f52b0-7663-4e2a-b940-79742bd20e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:46.957730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 915/1813] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:46 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:46.974575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392969a-ec4b-480b-90bc-cf0a24502b3f None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:46.985542 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1392969a-ec4b-480b-90bc-cf0a24502b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:47.083526 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f52b0-7663-4e2a-b940-79742bd20e17 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:47.093026 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384a7df-5ef6-4381-8a9f-09822e994eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:47.093499 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384a7df-5ef6-4381-8a9f-09822e994eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:47.093790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384a7df-5ef6-4381-8a9f-09822e994eed admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:47.094071 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384a7df-5ef6-4381-8a9f-09822e994eed admin admin] PATH_INFO: `/v3/projects/1a2d5828956b436ea162b749e3298006` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:47.101126 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384a7df-5ef6-4381-8a9f-09822e994eed admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a2d5828956b436ea162b749e3298006)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:47.103083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384a7df-5ef6-4381-8a9f-09822e994eed admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:47.109590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:47.191280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392969a-ec4b-480b-90bc-cf0a24502b3f None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:47.199994 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f8f3d-0793-44b3-bbd4-0c1da39ed51c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:47.203177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f8f3d-0793-44b3-bbd4-0c1da39ed51c service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:47.203177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f8f3d-0793-44b3-bbd4-0c1da39ed51c service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:47.203177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f8f3d-0793-44b3-bbd4-0c1da39ed51c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:47.217007 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b384a7df-5ef6-4381-8a9f-09822e994eed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a2d5828956b436ea162b749e3298006', 'request_id': 'req-b384a7df-5ef6-4381-8a9f-09822e994eed'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:47.232483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b384a7df-5ef6-4381-8a9f-09822e994eed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a2d5828956b436ea162b749e3298006', 'request_id': 'req-b384a7df-5ef6-4381-8a9f-09822e994eed'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:47.236346 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b384a7df-5ef6-4381-8a9f-09822e994eed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a2d5828956b436ea162b749e3298006', 'request_id': 'req-b384a7df-5ef6-4381-8a9f-09822e994eed'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:47.250356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f8f3d-0793-44b3-bbd4-0c1da39ed51c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:47.253514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f8f3d-0793-44b3-bbd4-0c1da39ed51c service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:47.266796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b384a7df-5ef6-4381-8a9f-09822e994eed admin admin] The token cache is being invalidate because project 1a2d5828956b436ea162b749e3298006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:47.267270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b384a7df-5ef6-4381-8a9f-09822e994eed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:47.268178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b384a7df-5ef6-4381-8a9f-09822e994eed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:47.269804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 899/1814] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:46 2022] DELETE /identity/v3/projects/1a2d5828956b436ea162b749e3298006 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:47.275620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 916/1815] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:46 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:47.276962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384a7df-5ef6-4381-8a9f-09822e994eed admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:47.290265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b384a7df-5ef6-4381-8a9f-09822e994eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:47.461013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f8f3d-0793-44b3-bbd4-0c1da39ed51c service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:47.462790 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-532f8f3d-0793-44b3-bbd4-0c1da39ed51c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:47.465268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384a7df-5ef6-4381-8a9f-09822e994eed admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:47.477270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0661af-b272-44be-8851-682fc8bf0db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:47.478056 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0661af-b272-44be-8851-682fc8bf0db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:47.478416 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0661af-b272-44be-8851-682fc8bf0db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:47.478780 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0661af-b272-44be-8851-682fc8bf0db8 admin admin] PATH_INFO: `/v3/users/c84338b02b474eaa9f02288d679da181` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:47.481815 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f8f3d-0793-44b3-bbd4-0c1da39ed51c service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:47.482516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0661af-b272-44be-8851-682fc8bf0db8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c84338b02b474eaa9f02288d679da181)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:47.485457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0661af-b272-44be-8851-682fc8bf0db8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:47.623282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-2a0661af-b272-44be-8851-682fc8bf0db8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c84338b02b474eaa9f02288d679da181'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:47.625719 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511e700-9dbb-4b8a-b2b6-be034b0be20f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:47.626656 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511e700-9dbb-4b8a-b2b6-be034b0be20f service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:47.627454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511e700-9dbb-4b8a-b2b6-be034b0be20f service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:47.627972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511e700-9dbb-4b8a-b2b6-be034b0be20f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:47.630603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-3511e700-9dbb-4b8a-b2b6-be034b0be20f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:47.650930 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-2a0661af-b272-44be-8851-682fc8bf0db8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c84338b02b474eaa9f02288d679da181'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:47.682951 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-2a0661af-b272-44be-8851-682fc8bf0db8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c84338b02b474eaa9f02288d679da181'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:47.704954 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-2a0661af-b272-44be-8851-682fc8bf0db8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c84338b02b474eaa9f02288d679da181'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:47.706149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-2a0661af-b272-44be-8851-682fc8bf0db8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c84338b02b474eaa9f02288d679da181'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:47.710312 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:47.731819 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 900/1816] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:47 2022] DELETE /identity/v3/users/c84338b02b474eaa9f02288d679da181 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:47.815682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511e700-9dbb-4b8a-b2b6-be034b0be20f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:47.819860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511e700-9dbb-4b8a-b2b6-be034b0be20f service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:47.835345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 917/1817] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:50:48.202165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0661af-b272-44be-8851-682fc8bf0db8 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:48.204194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-2a0661af-b272-44be-8851-682fc8bf0db8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:48.245558 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:48.390581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0661af-b272-44be-8851-682fc8bf0db8 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:48.406484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b5614-c3f1-46b5-91a1-550d27565891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:48.407771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b5614-c3f1-46b5-91a1-550d27565891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:48.408656 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b5614-c3f1-46b5-91a1-550d27565891 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:48.409211 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b5614-c3f1-46b5-91a1-550d27565891 admin admin] PATH_INFO: `/v3/projects/387e0625f71b4ace85f7659d71b3537f` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:48.412164 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b5614-c3f1-46b5-91a1-550d27565891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=387e0625f71b4ace85f7659d71b3537f)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:48.418647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b5614-c3f1-46b5-91a1-550d27565891 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:48.531412 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-448b5614-c3f1-46b5-91a1-550d27565891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '387e0625f71b4ace85f7659d71b3537f', 'request_id': 'req-448b5614-c3f1-46b5-91a1-550d27565891'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:48.549115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-448b5614-c3f1-46b5-91a1-550d27565891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '387e0625f71b4ace85f7659d71b3537f', 'request_id': 'req-448b5614-c3f1-46b5-91a1-550d27565891'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:48.566020 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-448b5614-c3f1-46b5-91a1-550d27565891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '387e0625f71b4ace85f7659d71b3537f', 'request_id': 'req-448b5614-c3f1-46b5-91a1-550d27565891'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:48.595731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-448b5614-c3f1-46b5-91a1-550d27565891 admin admin] The token cache is being invalidate because project 387e0625f71b4ace85f7659d71b3537f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:48.599970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-448b5614-c3f1-46b5-91a1-550d27565891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:48.601750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-448b5614-c3f1-46b5-91a1-550d27565891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:48.610958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 901/1818] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:48 2022] DELETE /identity/v3/projects/387e0625f71b4ace85f7659d71b3537f => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:48.612825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4506f-06f4-4363-97a2-273763fa6314 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:48.613340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4506f-06f4-4363-97a2-273763fa6314 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:48.613522 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4506f-06f4-4363-97a2-273763fa6314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:48.651980 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-8fc4506f-06f4-4363-97a2-273763fa6314 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:48.653348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8fc4506f-06f4-4363-97a2-273763fa6314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:48.785149 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 918/1819] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:48.799363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d61842-6110-4985-8ec2-07358098c6dd None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:48.799855 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d61842-6110-4985-8ec2-07358098c6dd None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:48.800733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d61842-6110-4985-8ec2-07358098c6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:48.823547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-04d61842-6110-4985-8ec2-07358098c6dd None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:48.824936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-04d61842-6110-4985-8ec2-07358098c6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:48.834113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 902/1820] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:48.838424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4506f-06f4-4363-97a2-273763fa6314 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:48.850443 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4506f-06f4-4363-97a2-273763fa6314 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:48.855213 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7df40-7150-44ef-a6cf-563fd65828fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:48.855839 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7df40-7150-44ef-a6cf-563fd65828fa admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:48.856114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7df40-7150-44ef-a6cf-563fd65828fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:48.856414 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7df40-7150-44ef-a6cf-563fd65828fa admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:48.857691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7df40-7150-44ef-a6cf-563fd65828fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:48.857691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7df40-7150-44ef-a6cf-563fd65828fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:48.859608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7df40-7150-44ef-a6cf-563fd65828fa admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:48.875569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 919/1821] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:48.882076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d61842-6110-4985-8ec2-07358098c6dd None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:48.892997 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d61842-6110-4985-8ec2-07358098c6dd None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:48.898213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfb26e-d7c5-49a2-a856-eb1bf83c680a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:48.899351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfb26e-d7c5-49a2-a856-eb1bf83c680a admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:48.899351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfb26e-d7c5-49a2-a856-eb1bf83c680a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:48.899569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfb26e-d7c5-49a2-a856-eb1bf83c680a admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:48.900071 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfb26e-d7c5-49a2-a856-eb1bf83c680a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:48.902171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfb26e-d7c5-49a2-a856-eb1bf83c680a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:48.935099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 903/1822] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:48 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:48.937495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7df40-7150-44ef-a6cf-563fd65828fa admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:48.947823 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7df40-7150-44ef-a6cf-563fd65828fa admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:49.008087 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d2a7b-0ba5-4f24-9bdb-fc355585bf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:49.008748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d2a7b-0ba5-4f24-9bdb-fc355585bf90 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:49.009059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d2a7b-0ba5-4f24-9bdb-fc355585bf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:49.009333 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d2a7b-0ba5-4f24-9bdb-fc355585bf90 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:49.010012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d2a7b-0ba5-4f24-9bdb-fc355585bf90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:49.012074 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d2a7b-0ba5-4f24-9bdb-fc355585bf90 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:49.035106 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 920/1823] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:48 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:50:49.039571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfb26e-d7c5-49a2-a856-eb1bf83c680a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:49.050346 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfb26e-d7c5-49a2-a856-eb1bf83c680a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:49.055294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c1e08-a140-424f-8f14-3a3642168e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:49.055918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c1e08-a140-424f-8f14-3a3642168e1f admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:49.056205 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c1e08-a140-424f-8f14-3a3642168e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:49.056528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c1e08-a140-424f-8f14-3a3642168e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:49.057229 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c1e08-a140-424f-8f14-3a3642168e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:49.058820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c1e08-a140-424f-8f14-3a3642168e1f admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:49.070079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 904/1824] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:49.074412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d2a7b-0ba5-4f24-9bdb-fc355585bf90 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:49.086405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d2a7b-0ba5-4f24-9bdb-fc355585bf90 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:49.091012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9202b-f862-4de7-83c8-dee974103353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:49.091619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9202b-f862-4de7-83c8-dee974103353 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:49.091931 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9202b-f862-4de7-83c8-dee974103353 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:49.092379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9202b-f862-4de7-83c8-dee974103353 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:49.093084 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9202b-f862-4de7-83c8-dee974103353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:49.094722 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9202b-f862-4de7-83c8-dee974103353 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:49.106871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 921/1825] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:49.111301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c1e08-a140-424f-8f14-3a3642168e1f admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:49.121514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c1e08-a140-424f-8f14-3a3642168e1f admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:49.127061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68b60d-7286-4228-86ed-7233b5f2ae60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:49.127953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68b60d-7286-4228-86ed-7233b5f2ae60 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:49.128498 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68b60d-7286-4228-86ed-7233b5f2ae60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:49.128783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68b60d-7286-4228-86ed-7233b5f2ae60 admin admin] PATH_INFO: `/v3/projects/7871ffa539134a17a32794b561e1ec29/users/1762b6d43e914ff68573875d8e1ffcdc/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:49.147323 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68b60d-7286-4228-86ed-7233b5f2ae60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7871ffa539134a17a32794b561e1ec29, user_id=1762b6d43e914ff68573875d8e1ffcdc, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:49.150074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68b60d-7286-4228-86ed-7233b5f2ae60 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:49.171317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 905/1826] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:49 2022] PUT /identity/v3/projects/7871ffa539134a17a32794b561e1ec29/users/1762b6d43e914ff68573875d8e1ffcdc/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:50:49.177700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb8262-4daa-4268-8d93-684ce01c89d2 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:49.177996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb8262-4daa-4268-8d93-684ce01c89d2 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:49.180141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb8262-4daa-4268-8d93-684ce01c89d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:49.216812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-53eb8262-4daa-4268-8d93-684ce01c89d2 None None] MFA Rules not processed for user `1762b6d43e914ff68573875d8e1ffcdc`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:49.218257 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-53eb8262-4daa-4268-8d93-684ce01c89d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:49.236556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:49.288405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 922/1827] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:49.293517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68b60d-7286-4228-86ed-7233b5f2ae60 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:49.335363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:49.359604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68b60d-7286-4228-86ed-7233b5f2ae60 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:49.364229 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572d3af-07cd-478d-b926-fe5a4f99b87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:49.364875 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572d3af-07cd-478d-b926-fe5a4f99b87e admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:49.365157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572d3af-07cd-478d-b926-fe5a4f99b87e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:49.365431 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572d3af-07cd-478d-b926-fe5a4f99b87e admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:49.366109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572d3af-07cd-478d-b926-fe5a4f99b87e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:49.367995 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572d3af-07cd-478d-b926-fe5a4f99b87e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:49.396777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 906/1828] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:49 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:50:49.400880 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb8262-4daa-4268-8d93-684ce01c89d2 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:49.412512 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb8262-4daa-4268-8d93-684ce01c89d2 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:49.479515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01812c-7bc5-47bc-a808-41d9ac683e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:49.479918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01812c-7bc5-47bc-a808-41d9ac683e33 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:49.480316 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01812c-7bc5-47bc-a808-41d9ac683e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:49.480730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01812c-7bc5-47bc-a808-41d9ac683e33 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:49.481517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01812c-7bc5-47bc-a808-41d9ac683e33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:49.483972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01812c-7bc5-47bc-a808-41d9ac683e33 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:49.509504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 923/1829] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:49 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:50:49.513207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572d3af-07cd-478d-b926-fe5a4f99b87e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:49.524279 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572d3af-07cd-478d-b926-fe5a4f99b87e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:49.528924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbd723-eac0-4b02-8f07-d70b8c1d21d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:49.529668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbd723-eac0-4b02-8f07-d70b8c1d21d2 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:49.530534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbd723-eac0-4b02-8f07-d70b8c1d21d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:49.530995 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbd723-eac0-4b02-8f07-d70b8c1d21d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:49.531953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbd723-eac0-4b02-8f07-d70b8c1d21d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:49.533703 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbd723-eac0-4b02-8f07-d70b8c1d21d2 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:49.550799 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 907/1830] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:50:49.555452 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01812c-7bc5-47bc-a808-41d9ac683e33 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:49.567924 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01812c-7bc5-47bc-a808-41d9ac683e33 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:49.572800 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f3ec4-38db-4501-b272-f905375a2b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:49.573544 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f3ec4-38db-4501-b272-f905375a2b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:49.573886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f3ec4-38db-4501-b272-f905375a2b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:49.574205 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f3ec4-38db-4501-b272-f905375a2b1a admin admin] PATH_INFO: `/v3/projects/d10cd3d5f89e4ad485b87efcba3de95d/users/118fa44ee22b43d4af2c8bc2e6299eb8/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:49.593849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f3ec4-38db-4501-b272-f905375a2b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d10cd3d5f89e4ad485b87efcba3de95d, user_id=118fa44ee22b43d4af2c8bc2e6299eb8, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:49.596700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f3ec4-38db-4501-b272-f905375a2b1a admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:49.611412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 924/1831] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:49 2022] PUT /identity/v3/projects/d10cd3d5f89e4ad485b87efcba3de95d/users/118fa44ee22b43d4af2c8bc2e6299eb8/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:49.615474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbd723-eac0-4b02-8f07-d70b8c1d21d2 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:49.626474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbd723-eac0-4b02-8f07-d70b8c1d21d2 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:49.683552 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011328d-f5bc-4715-a8a5-b4534145b279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:49.684086 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011328d-f5bc-4715-a8a5-b4534145b279 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:49.684414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011328d-f5bc-4715-a8a5-b4534145b279 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:49.684696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011328d-f5bc-4715-a8a5-b4534145b279 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:49.685388 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011328d-f5bc-4715-a8a5-b4534145b279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:49.687698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011328d-f5bc-4715-a8a5-b4534145b279 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:49.698319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 908/1832] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:49.704072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f3ec4-38db-4501-b272-f905375a2b1a admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:49.716037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f3ec4-38db-4501-b272-f905375a2b1a admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:49.725232 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38dcab-283a-49ce-b4e0-be70685ef33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:49.725953 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38dcab-283a-49ce-b4e0-be70685ef33c admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:49.726237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38dcab-283a-49ce-b4e0-be70685ef33c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:49.726513 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38dcab-283a-49ce-b4e0-be70685ef33c admin admin] PATH_INFO: `/v3/projects/d10cd3d5f89e4ad485b87efcba3de95d/users/118fa44ee22b43d4af2c8bc2e6299eb8/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:49.728509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38dcab-283a-49ce-b4e0-be70685ef33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d10cd3d5f89e4ad485b87efcba3de95d, user_id=118fa44ee22b43d4af2c8bc2e6299eb8, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:49.731347 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38dcab-283a-49ce-b4e0-be70685ef33c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:49.745599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 925/1833] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:49 2022] PUT /identity/v3/projects/d10cd3d5f89e4ad485b87efcba3de95d/users/118fa44ee22b43d4af2c8bc2e6299eb8/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:49.757099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10a0ab-2d04-48ff-9a4d-a6592fbca83f None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:49.757462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10a0ab-2d04-48ff-9a4d-a6592fbca83f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:49.757874 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10a0ab-2d04-48ff-9a4d-a6592fbca83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:49.789912 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-1c10a0ab-2d04-48ff-9a4d-a6592fbca83f None None] MFA Rules not processed for user `118fa44ee22b43d4af2c8bc2e6299eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:49.791132 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1c10a0ab-2d04-48ff-9a4d-a6592fbca83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:49.887689 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 909/1834] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:49.913223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38dcab-283a-49ce-b4e0-be70685ef33c admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:49.916755 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-aa38dcab-283a-49ce-b4e0-be70685ef33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:50.038567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38dcab-283a-49ce-b4e0-be70685ef33c admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:50.042560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404eebfa-f5d2-4607-9fa2-fed38e83fbde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:50.043285 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404eebfa-f5d2-4607-9fa2-fed38e83fbde service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:50.043680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404eebfa-f5d2-4607-9fa2-fed38e83fbde service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:50.044094 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404eebfa-f5d2-4607-9fa2-fed38e83fbde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:50.075548 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404eebfa-f5d2-4607-9fa2-fed38e83fbde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:50.077607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404eebfa-f5d2-4607-9fa2-fed38e83fbde service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:50.098580 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 926/1835] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:50.256500 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10a0ab-2d04-48ff-9a4d-a6592fbca83f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:50.259830 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1c10a0ab-2d04-48ff-9a4d-a6592fbca83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:50.285572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10a0ab-2d04-48ff-9a4d-a6592fbca83f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:50.297577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8078843-1433-4599-9e2f-e86dea7f9e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:50.298246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8078843-1433-4599-9e2f-e86dea7f9e12 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:50.298533 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8078843-1433-4599-9e2f-e86dea7f9e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:50.298811 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8078843-1433-4599-9e2f-e86dea7f9e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:50.367574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:50.443509 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8078843-1433-4599-9e2f-e86dea7f9e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:50.447010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8078843-1433-4599-9e2f-e86dea7f9e12 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:50.474194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 910/1836] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:50.996281 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd8c5e-6196-459d-97e6-a8f8196fc19f None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:50.997437 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd8c5e-6196-459d-97e6-a8f8196fc19f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:50.999544 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd8c5e-6196-459d-97e6-a8f8196fc19f None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:51.001892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 927/1837] 10.222.0.69 () {54 vars in 921 bytes} [Wed Aug 3 18:50:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:50:51.012615 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622af3b-8222-46b9-9f3b-c302f8f0851e None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:51.013049 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622af3b-8222-46b9-9f3b-c302f8f0851e None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:51.013228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622af3b-8222-46b9-9f3b-c302f8f0851e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:51.058511 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-4622af3b-8222-46b9-9f3b-c302f8f0851e None None] MFA Rules not processed for user `850a50cfe4394febbafb0908b08888b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:51.059727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4622af3b-8222-46b9-9f3b-c302f8f0851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:51.176515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 911/1838] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Aug 3 18:50:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:51.189861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd8c5e-6196-459d-97e6-a8f8196fc19f None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:51.191946 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-6ddd8c5e-6196-459d-97e6-a8f8196fc19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:51.217158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd8c5e-6196-459d-97e6-a8f8196fc19f None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:51.221665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484379d-8a89-4aed-82d9-d485865b3bae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:51.222257 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484379d-8a89-4aed-82d9-d485865b3bae service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:51.222541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484379d-8a89-4aed-82d9-d485865b3bae service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:51.222813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484379d-8a89-4aed-82d9-d485865b3bae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:51.232422 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484379d-8a89-4aed-82d9-d485865b3bae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:51.234501 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484379d-8a89-4aed-82d9-d485865b3bae service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:51.241537 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:51.247403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 928/1839] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:51.940500 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622af3b-8222-46b9-9f3b-c302f8f0851e None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:51.942517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4622af3b-8222-46b9-9f3b-c302f8f0851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:52.125474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622af3b-8222-46b9-9f3b-c302f8f0851e None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:52.141937 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163e072-4364-4b5f-b86e-bf2dee11623e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:52.145125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163e072-4364-4b5f-b86e-bf2dee11623e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:52.145694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163e072-4364-4b5f-b86e-bf2dee11623e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:52.146238 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163e072-4364-4b5f-b86e-bf2dee11623e admin admin] PATH_INFO: `/v3/users/2850912c2ac6417793a987a15334db28` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:52.149116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163e072-4364-4b5f-b86e-bf2dee11623e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2850912c2ac6417793a987a15334db28)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:52.156244 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163e072-4364-4b5f-b86e-bf2dee11623e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:52.291324 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3163e072-4364-4b5f-b86e-bf2dee11623e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2850912c2ac6417793a987a15334db28'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:52.311783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3163e072-4364-4b5f-b86e-bf2dee11623e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2850912c2ac6417793a987a15334db28'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:52.323184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:52.332442 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3163e072-4364-4b5f-b86e-bf2dee11623e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2850912c2ac6417793a987a15334db28'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:52.343694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3163e072-4364-4b5f-b86e-bf2dee11623e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2850912c2ac6417793a987a15334db28'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:52.344492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3163e072-4364-4b5f-b86e-bf2dee11623e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2850912c2ac6417793a987a15334db28'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:52.355222 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 912/1840] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:51 2022] DELETE /identity/v3/users/2850912c2ac6417793a987a15334db28 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:52.356807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9b4c4-77ef-4996-b812-ff2c130c2b25 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:52.358208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9b4c4-77ef-4996-b812-ff2c130c2b25 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:52.358891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9b4c4-77ef-4996-b812-ff2c130c2b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:52.397049 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-07f9b4c4-77ef-4996-b812-ff2c130c2b25 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:52.399559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-07f9b4c4-77ef-4996-b812-ff2c130c2b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:52.569379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 929/1841] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:50:52.694256 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163e072-4364-4b5f-b86e-bf2dee11623e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:52.695738 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-3163e072-4364-4b5f-b86e-bf2dee11623e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:52.812765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163e072-4364-4b5f-b86e-bf2dee11623e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:52.817750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124c280-d684-4cec-9931-00bad3833f8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:52.818264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124c280-d684-4cec-9931-00bad3833f8d service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:52.818555 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124c280-d684-4cec-9931-00bad3833f8d service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:52.818871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124c280-d684-4cec-9931-00bad3833f8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:52.845214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124c280-d684-4cec-9931-00bad3833f8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:52.846911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124c280-d684-4cec-9931-00bad3833f8d service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:52.859785 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 913/1842] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:53.565070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9b4c4-77ef-4996-b812-ff2c130c2b25 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:53.565615 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124c280-d684-4cec-9931-00bad3833f8d service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:53.567480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1124c280-d684-4cec-9931-00bad3833f8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:53.567910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-07f9b4c4-77ef-4996-b812-ff2c130c2b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:53.587769 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124c280-d684-4cec-9931-00bad3833f8d service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:53.597908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:53.598980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:53.599554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:53.599980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672 admin admin] PATH_INFO: `/v3/projects/95827f33b42c4789856e0918ebf02807` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:53.601647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95827f33b42c4789856e0918ebf02807)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:53.603850 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:53.610544 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9b4c4-77ef-4996-b812-ff2c130c2b25 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:53.616632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22a312-c7b7-4dde-8ac6-089d8a1dbf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:53.617299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22a312-c7b7-4dde-8ac6-089d8a1dbf8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:53.617583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22a312-c7b7-4dde-8ac6-089d8a1dbf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:53.617857 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22a312-c7b7-4dde-8ac6-089d8a1dbf8f admin admin] PATH_INFO: `/v3/users/3ee7c9eab128461782f95adf5cda294f` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:53.619701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22a312-c7b7-4dde-8ac6-089d8a1dbf8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ee7c9eab128461782f95adf5cda294f)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:53.621762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22a312-c7b7-4dde-8ac6-089d8a1dbf8f admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:53.696779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-2c22a312-c7b7-4dde-8ac6-089d8a1dbf8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ee7c9eab128461782f95adf5cda294f'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:53.706910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-2c22a312-c7b7-4dde-8ac6-089d8a1dbf8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ee7c9eab128461782f95adf5cda294f'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:53.723606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95827f33b42c4789856e0918ebf02807', 'request_id': 'req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:53.739276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-2c22a312-c7b7-4dde-8ac6-089d8a1dbf8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ee7c9eab128461782f95adf5cda294f'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:53.749412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-2c22a312-c7b7-4dde-8ac6-089d8a1dbf8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ee7c9eab128461782f95adf5cda294f'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:53.750476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95827f33b42c4789856e0918ebf02807', 'request_id': 'req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:53.750600 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-2c22a312-c7b7-4dde-8ac6-089d8a1dbf8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ee7c9eab128461782f95adf5cda294f'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:53.758355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95827f33b42c4789856e0918ebf02807', 'request_id': 'req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:53.762090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 930/1843] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:53 2022] DELETE /identity/v3/users/3ee7c9eab128461782f95adf5cda294f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:53.767752 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf437f-1037-4e47-9bec-1e20f96c7ec1 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:53.768135 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf437f-1037-4e47-9bec-1e20f96c7ec1 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:53.768617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf437f-1037-4e47-9bec-1e20f96c7ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:53.777014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672 admin admin] The token cache is being invalidate because project 95827f33b42c4789856e0918ebf02807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:53.777811 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:53.778662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:53.781679 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 914/1844] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:53 2022] DELETE /identity/v3/projects/95827f33b42c4789856e0918ebf02807 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:53.785823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:53.787381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:53.792639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-1faf437f-1037-4e47-9bec-1e20f96c7ec1 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:53.794339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1faf437f-1037-4e47-9bec-1e20f96c7ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:53.874644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 931/1845] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:53.886742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf437f-1037-4e47-9bec-1e20f96c7ec1 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:53.887858 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1faf437f-1037-4e47-9bec-1e20f96c7ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:53.895165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bd8ad-530c-4b2e-9ba5-e52c777a1672 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:53.900043 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7aa644-6d90-447a-9fa6-f75fd28df63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:53.900922 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7aa644-6d90-447a-9fa6-f75fd28df63c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:53.901215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7aa644-6d90-447a-9fa6-f75fd28df63c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:53.901502 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7aa644-6d90-447a-9fa6-f75fd28df63c admin admin] PATH_INFO: `/v3/users/f396c8dc1a264966be71e4a865b30940` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:53.902730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7aa644-6d90-447a-9fa6-f75fd28df63c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f396c8dc1a264966be71e4a865b30940)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:53.905279 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7aa644-6d90-447a-9fa6-f75fd28df63c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:53.916976 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:53.919723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:53.979718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-da7aa644-6d90-447a-9fa6-f75fd28df63c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f396c8dc1a264966be71e4a865b30940'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:53.992265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf437f-1037-4e47-9bec-1e20f96c7ec1 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:53.994564 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-da7aa644-6d90-447a-9fa6-f75fd28df63c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f396c8dc1a264966be71e4a865b30940'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:53.997985 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ee9d0-6e7d-4846-9f46-4043c0a92b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:53.999409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ee9d0-6e7d-4846-9f46-4043c0a92b7f service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:53.999703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ee9d0-6e7d-4846-9f46-4043c0a92b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:53.999988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ee9d0-6e7d-4846-9f46-4043c0a92b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:54.012588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-da7aa644-6d90-447a-9fa6-f75fd28df63c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f396c8dc1a264966be71e4a865b30940'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.015249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ee9d0-6e7d-4846-9f46-4043c0a92b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:54.017120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ee9d0-6e7d-4846-9f46-4043c0a92b7f service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:54.024903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-da7aa644-6d90-447a-9fa6-f75fd28df63c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f396c8dc1a264966be71e4a865b30940'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.025895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-da7aa644-6d90-447a-9fa6-f75fd28df63c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f396c8dc1a264966be71e4a865b30940'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.027951 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 932/1846] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:50:54.028956 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ee9d0-6e7d-4846-9f46-4043c0a92b7f service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:54.031354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4f6ee9d0-6e7d-4846-9f46-4043c0a92b7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:54.035598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 915/1847] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:53 2022] DELETE /identity/v3/users/f396c8dc1a264966be71e4a865b30940 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:50:54.063849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ee9d0-6e7d-4846-9f46-4043c0a92b7f service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:54.204086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306e2cf-18ec-496a-9c2e-7aad19c6baa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:54.207048 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306e2cf-18ec-496a-9c2e-7aad19c6baa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:54.209441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306e2cf-18ec-496a-9c2e-7aad19c6baa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:54.210169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306e2cf-18ec-496a-9c2e-7aad19c6baa0 admin admin] PATH_INFO: `/v3/users/552d619912cd45ccae9b9a6068b37e02` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:54.211570 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306e2cf-18ec-496a-9c2e-7aad19c6baa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=552d619912cd45ccae9b9a6068b37e02)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:54.217747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306e2cf-18ec-496a-9c2e-7aad19c6baa0 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:54.294082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b306e2cf-18ec-496a-9c2e-7aad19c6baa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '552d619912cd45ccae9b9a6068b37e02'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.304791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b306e2cf-18ec-496a-9c2e-7aad19c6baa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '552d619912cd45ccae9b9a6068b37e02'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.327614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b306e2cf-18ec-496a-9c2e-7aad19c6baa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '552d619912cd45ccae9b9a6068b37e02'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.337464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b306e2cf-18ec-496a-9c2e-7aad19c6baa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '552d619912cd45ccae9b9a6068b37e02'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.338753 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b306e2cf-18ec-496a-9c2e-7aad19c6baa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '552d619912cd45ccae9b9a6068b37e02'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.350148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 933/1848] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:54 2022] DELETE /identity/v3/users/552d619912cd45ccae9b9a6068b37e02 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:54.355121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c80583-9c64-40a8-a0d4-454097b16ca2 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:54.355121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c80583-9c64-40a8-a0d4-454097b16ca2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:54.355373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c80583-9c64-40a8-a0d4-454097b16ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:54.377451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-42c80583-9c64-40a8-a0d4-454097b16ca2 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:54.378793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-42c80583-9c64-40a8-a0d4-454097b16ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:54.452957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 916/1849] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:54.469937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306e2cf-18ec-496a-9c2e-7aad19c6baa0 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:54.471450 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b306e2cf-18ec-496a-9c2e-7aad19c6baa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:54.524549 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:54.557820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306e2cf-18ec-496a-9c2e-7aad19c6baa0 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:54.561905 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba11bd-48d6-4b51-95d6-7932ba81ae11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:54.562552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba11bd-48d6-4b51-95d6-7932ba81ae11 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:54.562844 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba11bd-48d6-4b51-95d6-7932ba81ae11 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:54.563123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba11bd-48d6-4b51-95d6-7932ba81ae11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:54.585659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba11bd-48d6-4b51-95d6-7932ba81ae11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:54.588026 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba11bd-48d6-4b51-95d6-7932ba81ae11 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:54.602135 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 934/1850] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:54.685578 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c80583-9c64-40a8-a0d4-454097b16ca2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:54.687786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-42c80583-9c64-40a8-a0d4-454097b16ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:54.700883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c80583-9c64-40a8-a0d4-454097b16ca2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:54.705637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba11bd-48d6-4b51-95d6-7932ba81ae11 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:54.708230 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-40ba11bd-48d6-4b51-95d6-7932ba81ae11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:54.709603 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728a97e-684a-423d-a862-48af95281bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:54.710679 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728a97e-684a-423d-a862-48af95281bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:54.711329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728a97e-684a-423d-a862-48af95281bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:54.711839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728a97e-684a-423d-a862-48af95281bb1 admin admin] PATH_INFO: `/v3/projects/4a2a6ea216704e7eaf2aa8f68fedab9c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:54.713528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728a97e-684a-423d-a862-48af95281bb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a2a6ea216704e7eaf2aa8f68fedab9c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:54.716035 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728a97e-684a-423d-a862-48af95281bb1 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:54.735038 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba11bd-48d6-4b51-95d6-7932ba81ae11 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:54.741027 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786beda-5539-4714-a5ae-8f0dd54e0f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:54.741802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786beda-5539-4714-a5ae-8f0dd54e0f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:54.742329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786beda-5539-4714-a5ae-8f0dd54e0f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:54.742753 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786beda-5539-4714-a5ae-8f0dd54e0f42 admin admin] PATH_INFO: `/v3/projects/99c3acb259d64e7ea2648cee0b2cd353` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:54.744584 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786beda-5539-4714-a5ae-8f0dd54e0f42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c3acb259d64e7ea2648cee0b2cd353)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:54.746717 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786beda-5539-4714-a5ae-8f0dd54e0f42 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:54.822116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1728a97e-684a-423d-a862-48af95281bb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a2a6ea216704e7eaf2aa8f68fedab9c', 'request_id': 'req-1728a97e-684a-423d-a862-48af95281bb1'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.830282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1728a97e-684a-423d-a862-48af95281bb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a2a6ea216704e7eaf2aa8f68fedab9c', 'request_id': 'req-1728a97e-684a-423d-a862-48af95281bb1'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.833117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0786beda-5539-4714-a5ae-8f0dd54e0f42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c3acb259d64e7ea2648cee0b2cd353', 'request_id': 'req-0786beda-5539-4714-a5ae-8f0dd54e0f42'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.839624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1728a97e-684a-423d-a862-48af95281bb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a2a6ea216704e7eaf2aa8f68fedab9c', 'request_id': 'req-1728a97e-684a-423d-a862-48af95281bb1'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.841100 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0786beda-5539-4714-a5ae-8f0dd54e0f42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c3acb259d64e7ea2648cee0b2cd353', 'request_id': 'req-0786beda-5539-4714-a5ae-8f0dd54e0f42'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.860144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0786beda-5539-4714-a5ae-8f0dd54e0f42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c3acb259d64e7ea2648cee0b2cd353', 'request_id': 'req-0786beda-5539-4714-a5ae-8f0dd54e0f42'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.867940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1728a97e-684a-423d-a862-48af95281bb1 admin admin] The token cache is being invalidate because project 4a2a6ea216704e7eaf2aa8f68fedab9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:54.868267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1728a97e-684a-423d-a862-48af95281bb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.869150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1728a97e-684a-423d-a862-48af95281bb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.876437 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 917/1851] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:54 2022] DELETE /identity/v3/projects/4a2a6ea216704e7eaf2aa8f68fedab9c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:54.876437 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728a97e-684a-423d-a862-48af95281bb1 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:54.879529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1728a97e-684a-423d-a862-48af95281bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:54.883180 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0786beda-5539-4714-a5ae-8f0dd54e0f42 admin admin] The token cache is being invalidate because project 99c3acb259d64e7ea2648cee0b2cd353 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:54.883476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0786beda-5539-4714-a5ae-8f0dd54e0f42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.885039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0786beda-5539-4714-a5ae-8f0dd54e0f42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:54.887450 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 935/1852] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:54 2022] DELETE /identity/v3/projects/99c3acb259d64e7ea2648cee0b2cd353 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:54.891422 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba90ed2-fba5-45fc-b54b-b2567764100f None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:54.894391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba90ed2-fba5-45fc-b54b-b2567764100f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:54.894548 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba90ed2-fba5-45fc-b54b-b2567764100f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:54.911612 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:54.924564 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-0ba90ed2-fba5-45fc-b54b-b2567764100f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:54.926496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0ba90ed2-fba5-45fc-b54b-b2567764100f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:54.985686 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728a97e-684a-423d-a862-48af95281bb1 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:55.008075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 936/1853] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:55.009136 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba90ed2-fba5-45fc-b54b-b2567764100f None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:55.010565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0ba90ed2-fba5-45fc-b54b-b2567764100f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:55.032615 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba90ed2-fba5-45fc-b54b-b2567764100f None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:55.038694 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23077187-eba2-4722-8939-285c3347d088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:55.039361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23077187-eba2-4722-8939-285c3347d088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:55.039636 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23077187-eba2-4722-8939-285c3347d088 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:55.040371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23077187-eba2-4722-8939-285c3347d088 admin admin] PATH_INFO: `/v3/users/0e203b8f85754bcbb878b09a2501b95b` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:55.041576 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23077187-eba2-4722-8939-285c3347d088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e203b8f85754bcbb878b09a2501b95b)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:55.043614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23077187-eba2-4722-8939-285c3347d088 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:55.061292 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9868bb5-4a7a-493a-b84c-dc26cdaefb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:55.061877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9868bb5-4a7a-493a-b84c-dc26cdaefb4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:55.062265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9868bb5-4a7a-493a-b84c-dc26cdaefb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:55.063411 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9868bb5-4a7a-493a-b84c-dc26cdaefb4b admin admin] PATH_INFO: `/v3/users/1762b6d43e914ff68573875d8e1ffcdc` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:55.064833 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9868bb5-4a7a-493a-b84c-dc26cdaefb4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1762b6d43e914ff68573875d8e1ffcdc)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:55.067486 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9868bb5-4a7a-493a-b84c-dc26cdaefb4b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:55.107631 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-23077187-eba2-4722-8939-285c3347d088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e203b8f85754bcbb878b09a2501b95b'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.129790 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-23077187-eba2-4722-8939-285c3347d088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e203b8f85754bcbb878b09a2501b95b'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.132273 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:55.139941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f9868bb5-4a7a-493a-b84c-dc26cdaefb4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1762b6d43e914ff68573875d8e1ffcdc'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.149791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f9868bb5-4a7a-493a-b84c-dc26cdaefb4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1762b6d43e914ff68573875d8e1ffcdc'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.153417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-23077187-eba2-4722-8939-285c3347d088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e203b8f85754bcbb878b09a2501b95b'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.161785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-23077187-eba2-4722-8939-285c3347d088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e203b8f85754bcbb878b09a2501b95b'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.162534 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-23077187-eba2-4722-8939-285c3347d088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e203b8f85754bcbb878b09a2501b95b'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.166101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f9868bb5-4a7a-493a-b84c-dc26cdaefb4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1762b6d43e914ff68573875d8e1ffcdc'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.168595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 937/1854] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:55 2022] DELETE /identity/v3/users/0e203b8f85754bcbb878b09a2501b95b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:55.170161 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfacf9-2f2c-4de7-bc6e-5817fadc2b8d None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:55.170454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfacf9-2f2c-4de7-bc6e-5817fadc2b8d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:55.170767 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfacf9-2f2c-4de7-bc6e-5817fadc2b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:55.184680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f9868bb5-4a7a-493a-b84c-dc26cdaefb4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1762b6d43e914ff68573875d8e1ffcdc'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.187017 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f9868bb5-4a7a-493a-b84c-dc26cdaefb4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1762b6d43e914ff68573875d8e1ffcdc'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.197495 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 918/1855] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:54 2022] DELETE /identity/v3/users/1762b6d43e914ff68573875d8e1ffcdc => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:55.203057 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b71fe5-b6f4-46e8-91fb-dae40585f06b None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:55.203503 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b71fe5-b6f4-46e8-91fb-dae40585f06b None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:55.203925 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b71fe5-b6f4-46e8-91fb-dae40585f06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:55.210295 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-c3cfacf9-2f2c-4de7-bc6e-5817fadc2b8d None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:55.212229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c3cfacf9-2f2c-4de7-bc6e-5817fadc2b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:55.230205 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-65b71fe5-b6f4-46e8-91fb-dae40585f06b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:55.231714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-65b71fe5-b6f4-46e8-91fb-dae40585f06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:55.326726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:55.337168 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 938/1856] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:55.339133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfacf9-2f2c-4de7-bc6e-5817fadc2b8d None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:55.340794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c3cfacf9-2f2c-4de7-bc6e-5817fadc2b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:55.360140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfacf9-2f2c-4de7-bc6e-5817fadc2b8d None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:55.365298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 919/1857] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:55.366249 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b71fe5-b6f4-46e8-91fb-dae40585f06b None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:55.380132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb079a7-9a9d-4b2e-a64c-358b228d748e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:55.380132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb079a7-9a9d-4b2e-a64c-358b228d748e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:55.380132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb079a7-9a9d-4b2e-a64c-358b228d748e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:55.380132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb079a7-9a9d-4b2e-a64c-358b228d748e admin admin] PATH_INFO: `/v3/projects/09b727efe3a0433aa7f53735a9bf9b04` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:55.380722 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b71fe5-b6f4-46e8-91fb-dae40585f06b None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:55.380823 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb079a7-9a9d-4b2e-a64c-358b228d748e admin admin] RBAC: Authorizing `identity:delete_project(project_id=09b727efe3a0433aa7f53735a9bf9b04)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:55.383815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6743356-99d9-4f07-816a-5ee3cdd61526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:55.384566 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6743356-99d9-4f07-816a-5ee3cdd61526 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:55.384918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6743356-99d9-4f07-816a-5ee3cdd61526 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:55.385406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6743356-99d9-4f07-816a-5ee3cdd61526 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:55.386231 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6743356-99d9-4f07-816a-5ee3cdd61526 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:55.386628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6743356-99d9-4f07-816a-5ee3cdd61526 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:55.389216 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6743356-99d9-4f07-816a-5ee3cdd61526 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:55.390972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb079a7-9a9d-4b2e-a64c-358b228d748e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:55.402919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 920/1858] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:55.403868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6743356-99d9-4f07-816a-5ee3cdd61526 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:55.410410 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f6743356-99d9-4f07-816a-5ee3cdd61526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:55.462419 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-edb079a7-9a9d-4b2e-a64c-358b228d748e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09b727efe3a0433aa7f53735a9bf9b04', 'request_id': 'req-edb079a7-9a9d-4b2e-a64c-358b228d748e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.469714 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-edb079a7-9a9d-4b2e-a64c-358b228d748e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09b727efe3a0433aa7f53735a9bf9b04', 'request_id': 'req-edb079a7-9a9d-4b2e-a64c-358b228d748e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.483361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-edb079a7-9a9d-4b2e-a64c-358b228d748e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09b727efe3a0433aa7f53735a9bf9b04', 'request_id': 'req-edb079a7-9a9d-4b2e-a64c-358b228d748e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.488463 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6743356-99d9-4f07-816a-5ee3cdd61526 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:55.503052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-edb079a7-9a9d-4b2e-a64c-358b228d748e admin admin] The token cache is being invalidate because project 09b727efe3a0433aa7f53735a9bf9b04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:55.503351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-edb079a7-9a9d-4b2e-a64c-358b228d748e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.504304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-edb079a7-9a9d-4b2e-a64c-358b228d748e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.508693 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 939/1859] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:55 2022] DELETE /identity/v3/projects/09b727efe3a0433aa7f53735a9bf9b04 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:55.511817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb079a7-9a9d-4b2e-a64c-358b228d748e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:55.512790 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-edb079a7-9a9d-4b2e-a64c-358b228d748e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:55.569038 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71de8a-b361-4059-acfb-5046310d1f85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:55.569977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71de8a-b361-4059-acfb-5046310d1f85 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:55.570374 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71de8a-b361-4059-acfb-5046310d1f85 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:55.571019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71de8a-b361-4059-acfb-5046310d1f85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:55.573061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-3b71de8a-b361-4059-acfb-5046310d1f85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:55.609944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:55.615583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb079a7-9a9d-4b2e-a64c-358b228d748e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:55.639833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11c197-45a9-4942-bb61-5d6348a6eb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:55.641023 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11c197-45a9-4942-bb61-5d6348a6eb53 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:55.641914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11c197-45a9-4942-bb61-5d6348a6eb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:55.642357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11c197-45a9-4942-bb61-5d6348a6eb53 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:55.643437 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11c197-45a9-4942-bb61-5d6348a6eb53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:55.645920 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11c197-45a9-4942-bb61-5d6348a6eb53 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:55.657094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71de8a-b361-4059-acfb-5046310d1f85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:55.658623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71de8a-b361-4059-acfb-5046310d1f85 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:55.671620 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 921/1860] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:55.672812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71de8a-b361-4059-acfb-5046310d1f85 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:55.674693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-3b71de8a-b361-4059-acfb-5046310d1f85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:55.687308 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 940/1861] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:55 2022] POST /identity/v3/projects => generated 363 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:55.687308 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11c197-45a9-4942-bb61-5d6348a6eb53 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:55.688049 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-6f11c197-45a9-4942-bb61-5d6348a6eb53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:55.691418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71de8a-b361-4059-acfb-5046310d1f85 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:55.727616 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:55.776873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7f6af-7f95-4808-a07b-73f38024109c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:55.777697 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7f6af-7f95-4808-a07b-73f38024109c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:55.780198 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7f6af-7f95-4808-a07b-73f38024109c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:55.780697 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7f6af-7f95-4808-a07b-73f38024109c admin admin] PATH_INFO: `/v3/users/878c2c4db7724196a617303577925c6b` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:55.784143 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7f6af-7f95-4808-a07b-73f38024109c admin admin] RBAC: Authorizing `identity:delete_user(user_id=878c2c4db7724196a617303577925c6b)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:55.786214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7f6af-7f95-4808-a07b-73f38024109c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:55.805568 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11c197-45a9-4942-bb61-5d6348a6eb53 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:55.812841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9644d-e50f-4c8f-adde-dd8a205131b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:55.813518 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9644d-e50f-4c8f-adde-dd8a205131b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:55.813883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9644d-e50f-4c8f-adde-dd8a205131b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:55.814166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9644d-e50f-4c8f-adde-dd8a205131b5 admin admin] PATH_INFO: `/v3/projects/a3b2f2b097c94b84ab2733501432ec32` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:55.815792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9644d-e50f-4c8f-adde-dd8a205131b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3b2f2b097c94b84ab2733501432ec32)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:55.818538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9644d-e50f-4c8f-adde-dd8a205131b5 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:55.886206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e8a7f6af-7f95-4808-a07b-73f38024109c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '878c2c4db7724196a617303577925c6b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.901394 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b7b9644d-e50f-4c8f-adde-dd8a205131b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3b2f2b097c94b84ab2733501432ec32', 'request_id': 'req-b7b9644d-e50f-4c8f-adde-dd8a205131b5'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.916259 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b7b9644d-e50f-4c8f-adde-dd8a205131b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3b2f2b097c94b84ab2733501432ec32', 'request_id': 'req-b7b9644d-e50f-4c8f-adde-dd8a205131b5'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.917961 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e8a7f6af-7f95-4808-a07b-73f38024109c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '878c2c4db7724196a617303577925c6b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.925412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b7b9644d-e50f-4c8f-adde-dd8a205131b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3b2f2b097c94b84ab2733501432ec32', 'request_id': 'req-b7b9644d-e50f-4c8f-adde-dd8a205131b5'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.944629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b7b9644d-e50f-4c8f-adde-dd8a205131b5 admin admin] The token cache is being invalidate because project a3b2f2b097c94b84ab2733501432ec32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:55.945028 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b7b9644d-e50f-4c8f-adde-dd8a205131b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.945771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b7b9644d-e50f-4c8f-adde-dd8a205131b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.946055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e8a7f6af-7f95-4808-a07b-73f38024109c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '878c2c4db7724196a617303577925c6b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.947346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 941/1862] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:55 2022] DELETE /identity/v3/projects/a3b2f2b097c94b84ab2733501432ec32 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:55.948312 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9644d-e50f-4c8f-adde-dd8a205131b5 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:55.951116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b7b9644d-e50f-4c8f-adde-dd8a205131b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:55.960721 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e8a7f6af-7f95-4808-a07b-73f38024109c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '878c2c4db7724196a617303577925c6b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.962493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e8a7f6af-7f95-4808-a07b-73f38024109c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '878c2c4db7724196a617303577925c6b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:55.971824 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 922/1863] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:55 2022] DELETE /identity/v3/users/878c2c4db7724196a617303577925c6b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:55.972857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c7f7b-56b3-4feb-90a0-118e21615d02 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:55.973149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c7f7b-56b3-4feb-90a0-118e21615d02 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:55.973426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c7f7b-56b3-4feb-90a0-118e21615d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:56.000936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f02c7f7b-56b3-4feb-90a0-118e21615d02 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:56.002570 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f02c7f7b-56b3-4feb-90a0-118e21615d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:56.043205 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9644d-e50f-4c8f-adde-dd8a205131b5 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:56.068421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:56.094825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 923/1864] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:56.107054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46417974-2f00-4a07-a8b5-c3ce8993c94a None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:56.107344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46417974-2f00-4a07-a8b5-c3ce8993c94a None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:56.108690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46417974-2f00-4a07-a8b5-c3ce8993c94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:56.137475 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5aba8-eb47-4f3d-aa45-f60186460f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:56.138371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5aba8-eb47-4f3d-aa45-f60186460f86 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:56.139355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5aba8-eb47-4f3d-aa45-f60186460f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:56.139581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-46417974-2f00-4a07-a8b5-c3ce8993c94a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:56.140912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5aba8-eb47-4f3d-aa45-f60186460f86 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:56.141957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-46417974-2f00-4a07-a8b5-c3ce8993c94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:56.143492 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5aba8-eb47-4f3d-aa45-f60186460f86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:56.146374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5aba8-eb47-4f3d-aa45-f60186460f86 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:56.161581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 924/1865] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:56.163170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46417974-2f00-4a07-a8b5-c3ce8993c94a None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:56.178027 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46417974-2f00-4a07-a8b5-c3ce8993c94a None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:56.183045 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 942/1866] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:55 2022] POST /identity/v3/users => generated 282 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:56.188180 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5aba8-eb47-4f3d-aa45-f60186460f86 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:56.188540 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb43a6f-0c8f-4398-9b20-5e90f5292c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:56.189236 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb43a6f-0c8f-4398-9b20-5e90f5292c2a admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:56.189610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb43a6f-0c8f-4398-9b20-5e90f5292c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:56.189885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb43a6f-0c8f-4398-9b20-5e90f5292c2a admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:56.190792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb43a6f-0c8f-4398-9b20-5e90f5292c2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:56.191110 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb43a6f-0c8f-4398-9b20-5e90f5292c2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:56.191443 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-3df5aba8-eb47-4f3d-aa45-f60186460f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:56.193046 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb43a6f-0c8f-4398-9b20-5e90f5292c2a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:56.208573 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5aba8-eb47-4f3d-aa45-f60186460f86 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:56.215167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 925/1867] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:56.215330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c1438-d175-464f-9063-e2ef5903d5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:56.215330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c1438-d175-464f-9063-e2ef5903d5cd admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:56.215330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c1438-d175-464f-9063-e2ef5903d5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:56.215614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c1438-d175-464f-9063-e2ef5903d5cd admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:56.216402 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c1438-d175-464f-9063-e2ef5903d5cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:56.216596 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb43a6f-0c8f-4398-9b20-5e90f5292c2a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:56.218809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c1438-d175-464f-9063-e2ef5903d5cd admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:56.234391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 943/1868] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:50:56.235889 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c1438-d175-464f-9063-e2ef5903d5cd admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:56.240027 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb43a6f-0c8f-4398-9b20-5e90f5292c2a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:56.251428 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c1438-d175-464f-9063-e2ef5903d5cd admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:56.258842 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c0e1e-09b6-4df8-804d-42bddfc65bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:56.259786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c0e1e-09b6-4df8-804d-42bddfc65bdf admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:56.260230 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c0e1e-09b6-4df8-804d-42bddfc65bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:56.260670 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d46fe8-56e6-47ce-b381-a4a62ca79a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:56.261255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c0e1e-09b6-4df8-804d-42bddfc65bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:56.262107 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c0e1e-09b6-4df8-804d-42bddfc65bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:56.264217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c0e1e-09b6-4df8-804d-42bddfc65bdf admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:56.266474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d46fe8-56e6-47ce-b381-a4a62ca79a73 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:56.267763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d46fe8-56e6-47ce-b381-a4a62ca79a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:56.268671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d46fe8-56e6-47ce-b381-a4a62ca79a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:56.270602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d46fe8-56e6-47ce-b381-a4a62ca79a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:56.276099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d46fe8-56e6-47ce-b381-a4a62ca79a73 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:56.304459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 944/1869] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:50:56.305431 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c0e1e-09b6-4df8-804d-42bddfc65bdf admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:56.317910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c0e1e-09b6-4df8-804d-42bddfc65bdf admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:56.323585 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37fdf6-c4c6-49af-a315-445cf43c56e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:56.325781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37fdf6-c4c6-49af-a315-445cf43c56e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:56.325781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37fdf6-c4c6-49af-a315-445cf43c56e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:56.325781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37fdf6-c4c6-49af-a315-445cf43c56e9 admin admin] PATH_INFO: `/v3/projects/ed690fe3c501418dbe3703cccc3e17d2/users/f0e472160e67460aa8fc724ed271e4f6/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:56.349449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37fdf6-c4c6-49af-a315-445cf43c56e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed690fe3c501418dbe3703cccc3e17d2, user_id=f0e472160e67460aa8fc724ed271e4f6, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:56.357178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37fdf6-c4c6-49af-a315-445cf43c56e9 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:56.363193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 926/1870] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:56 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:56.367698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d46fe8-56e6-47ce-b381-a4a62ca79a73 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:56.386531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:56.402242 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 945/1871] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:56 2022] PUT /identity/v3/projects/ed690fe3c501418dbe3703cccc3e17d2/users/f0e472160e67460aa8fc724ed271e4f6/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:56.418551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fc451-8c47-48c1-a806-f588143fdcdf None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:56.419030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fc451-8c47-48c1-a806-f588143fdcdf None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:56.419693 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fc451-8c47-48c1-a806-f588143fdcdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:56.461707 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-842fc451-8c47-48c1-a806-f588143fdcdf None None] MFA Rules not processed for user `f0e472160e67460aa8fc724ed271e4f6`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:56.463033 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-842fc451-8c47-48c1-a806-f588143fdcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:56.516684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d46fe8-56e6-47ce-b381-a4a62ca79a73 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:56.564705 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 946/1872] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:50:56.568393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fc451-8c47-48c1-a806-f588143fdcdf None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:56.613577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc14ba4-9bec-4c75-9a22-86fa9cc7f849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:56.614237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc14ba4-9bec-4c75-9a22-86fa9cc7f849 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:56.614584 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc14ba4-9bec-4c75-9a22-86fa9cc7f849 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:56.615434 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc14ba4-9bec-4c75-9a22-86fa9cc7f849 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:56.616268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc14ba4-9bec-4c75-9a22-86fa9cc7f849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:56.619852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc14ba4-9bec-4c75-9a22-86fa9cc7f849 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:56.640628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:56.656851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 927/1873] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:56 2022] POST /identity/v3/users => generated 294 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:50:56.657983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fc451-8c47-48c1-a806-f588143fdcdf None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:56.663042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc14ba4-9bec-4c75-9a22-86fa9cc7f849 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:56.665176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bece2-6b28-400c-8941-5d1e33765a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:56.666368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bece2-6b28-400c-8941-5d1e33765a1c admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:56.667075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bece2-6b28-400c-8941-5d1e33765a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:56.667673 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bece2-6b28-400c-8941-5d1e33765a1c admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:56.668757 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bece2-6b28-400c-8941-5d1e33765a1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:56.672399 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bece2-6b28-400c-8941-5d1e33765a1c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:56.684000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc14ba4-9bec-4c75-9a22-86fa9cc7f849 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:56.695099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a4b03-dca9-4f1a-9102-769f3caac107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:56.696620 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a4b03-dca9-4f1a-9102-769f3caac107 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:56.697107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a4b03-dca9-4f1a-9102-769f3caac107 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:56.698117 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a4b03-dca9-4f1a-9102-769f3caac107 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:56.699189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a4b03-dca9-4f1a-9102-769f3caac107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:56.702046 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a4b03-dca9-4f1a-9102-769f3caac107 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:56.715963 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 928/1874] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:50:56.719995 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a4b03-dca9-4f1a-9102-769f3caac107 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:56.730067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 947/1875] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:56 2022] POST /identity/v3/projects => generated 365 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:56.734579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bece2-6b28-400c-8941-5d1e33765a1c admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:56.740751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a4b03-dca9-4f1a-9102-769f3caac107 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:56.860185 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bece2-6b28-400c-8941-5d1e33765a1c admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:56.866311 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791ef0e-5d35-4e8c-94b4-2bb70d94d074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:56.866977 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791ef0e-5d35-4e8c-94b4-2bb70d94d074 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:56.867260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791ef0e-5d35-4e8c-94b4-2bb70d94d074 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:56.867539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791ef0e-5d35-4e8c-94b4-2bb70d94d074 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:56.868799 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791ef0e-5d35-4e8c-94b4-2bb70d94d074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:56.871053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791ef0e-5d35-4e8c-94b4-2bb70d94d074 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:56.910299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 948/1876] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:56 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:56.917868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791ef0e-5d35-4e8c-94b4-2bb70d94d074 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:56.926880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bff38-d1cf-46bb-93be-700d2a4ed05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:56.927567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bff38-d1cf-46bb-93be-700d2a4ed05b admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:56.927941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bff38-d1cf-46bb-93be-700d2a4ed05b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:56.928223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bff38-d1cf-46bb-93be-700d2a4ed05b admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:56.929083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2bff38-d1cf-46bb-93be-700d2a4ed05b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:56.930721 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2bff38-d1cf-46bb-93be-700d2a4ed05b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:56.942124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791ef0e-5d35-4e8c-94b4-2bb70d94d074 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:56.951005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 929/1877] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:56.955737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e03c1-9e36-41f8-9103-32eac6dc1d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:56.956183 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bff38-d1cf-46bb-93be-700d2a4ed05b admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:56.956560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e03c1-9e36-41f8-9103-32eac6dc1d7a admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:56.956992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e03c1-9e36-41f8-9103-32eac6dc1d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:56.957285 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e03c1-9e36-41f8-9103-32eac6dc1d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:56.958974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e03c1-9e36-41f8-9103-32eac6dc1d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:56.961622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e03c1-9e36-41f8-9103-32eac6dc1d7a admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:56.981271 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 949/1878] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:56.982126 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bff38-d1cf-46bb-93be-700d2a4ed05b admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:56.983974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e03c1-9e36-41f8-9103-32eac6dc1d7a admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:56.988982 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750e297-03ff-45c4-b749-1de61e6e5635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:56.989907 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750e297-03ff-45c4-b749-1de61e6e5635 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:56.990363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750e297-03ff-45c4-b749-1de61e6e5635 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:56.990697 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750e297-03ff-45c4-b749-1de61e6e5635 admin admin] PATH_INFO: `/v3/projects/1cfdb70f6c034cfbaf6805e2b33e3c71/users/49f2a958c5394fc5877fc468f0916d34/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:57.004003 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e03c1-9e36-41f8-9103-32eac6dc1d7a admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:57.009386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750e297-03ff-45c4-b749-1de61e6e5635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cfdb70f6c034cfbaf6805e2b33e3c71, user_id=49f2a958c5394fc5877fc468f0916d34, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:57.013188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750e297-03ff-45c4-b749-1de61e6e5635 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:57.013567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd51358-b928-46ff-834d-32f81cc35627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:57.014601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd51358-b928-46ff-834d-32f81cc35627 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:57.015237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd51358-b928-46ff-834d-32f81cc35627 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:57.015829 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd51358-b928-46ff-834d-32f81cc35627 admin admin] PATH_INFO: `/v3/projects/aef95145859544428e263817cfdd9c56/users/bd014fe75d384061a2e56e875401f4d9/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:57.039793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 930/1879] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:56 2022] PUT /identity/v3/projects/1cfdb70f6c034cfbaf6805e2b33e3c71/users/49f2a958c5394fc5877fc468f0916d34/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:57.051181 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c06ae-295c-4d59-8f3f-adb8bdbec8f2 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:57.051517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c06ae-295c-4d59-8f3f-adb8bdbec8f2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:57.051639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd51358-b928-46ff-834d-32f81cc35627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aef95145859544428e263817cfdd9c56, user_id=bd014fe75d384061a2e56e875401f4d9, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:57.051731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c06ae-295c-4d59-8f3f-adb8bdbec8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:57.055030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd51358-b928-46ff-834d-32f81cc35627 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:57.077801 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:57.092949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 950/1880] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:56 2022] PUT /identity/v3/projects/aef95145859544428e263817cfdd9c56/users/bd014fe75d384061a2e56e875401f4d9/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:57.097906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd51358-b928-46ff-834d-32f81cc35627 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:57.117577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-bd7c06ae-295c-4d59-8f3f-adb8bdbec8f2 None None] MFA Rules not processed for user `49f2a958c5394fc5877fc468f0916d34`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:57.119139 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-bd7c06ae-295c-4d59-8f3f-adb8bdbec8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:57.213988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd51358-b928-46ff-834d-32f81cc35627 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:57.224426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0a08c-6e98-4a95-99a7-9791f5fae9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:57.224793 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0a08c-6e98-4a95-99a7-9791f5fae9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:57.225308 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0a08c-6e98-4a95-99a7-9791f5fae9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:57.225806 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0a08c-6e98-4a95-99a7-9791f5fae9d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:57.227144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0a08c-6e98-4a95-99a7-9791f5fae9d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:57.240554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0a08c-6e98-4a95-99a7-9791f5fae9d5 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:57.294361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 951/1881] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:57.298620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0a08c-6e98-4a95-99a7-9791f5fae9d5 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:57.318228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 931/1882] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:50:57.325246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c06ae-295c-4d59-8f3f-adb8bdbec8f2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:57.329382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0a08c-6e98-4a95-99a7-9791f5fae9d5 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:57.337751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba143156-a8de-47f1-9d37-cfa557beb21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:57.339125 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba143156-a8de-47f1-9d37-cfa557beb21c admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:57.339729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba143156-a8de-47f1-9d37-cfa557beb21c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:57.340400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba143156-a8de-47f1-9d37-cfa557beb21c admin admin] PATH_INFO: `/v3/projects/aef95145859544428e263817cfdd9c56/users/bd014fe75d384061a2e56e875401f4d9/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:57.344112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba143156-a8de-47f1-9d37-cfa557beb21c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aef95145859544428e263817cfdd9c56, user_id=bd014fe75d384061a2e56e875401f4d9, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:57.351508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba143156-a8de-47f1-9d37-cfa557beb21c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:57.369395 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 952/1883] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:57 2022] PUT /identity/v3/projects/aef95145859544428e263817cfdd9c56/users/bd014fe75d384061a2e56e875401f4d9/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:50:57.386421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35dd4a-f4b2-49a2-bb03-c6030d8d4295 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:57.386712 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35dd4a-f4b2-49a2-bb03-c6030d8d4295 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:57.386993 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35dd4a-f4b2-49a2-bb03-c6030d8d4295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:57.392126 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c06ae-295c-4d59-8f3f-adb8bdbec8f2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:57.450046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-cf35dd4a-f4b2-49a2-bb03-c6030d8d4295 None None] MFA Rules not processed for user `bd014fe75d384061a2e56e875401f4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:57.452431 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-cf35dd4a-f4b2-49a2-bb03-c6030d8d4295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:57.457074 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:57.543869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43700ab-4289-4a56-ad7b-e99f274145aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:57.548628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43700ab-4289-4a56-ad7b-e99f274145aa admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:57.550677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43700ab-4289-4a56-ad7b-e99f274145aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:57.551607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43700ab-4289-4a56-ad7b-e99f274145aa admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:57.553210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43700ab-4289-4a56-ad7b-e99f274145aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:57.557027 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43700ab-4289-4a56-ad7b-e99f274145aa admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:57.611141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 953/1884] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:50:57.611141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35dd4a-f4b2-49a2-bb03-c6030d8d4295 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:57.611819 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 932/1885] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:57 2022] POST /identity/v3/projects => generated 385 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:57.613504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-cf35dd4a-f4b2-49a2-bb03-c6030d8d4295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:57.619572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43700ab-4289-4a56-ad7b-e99f274145aa admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:57.636039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35dd4a-f4b2-49a2-bb03-c6030d8d4295 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:57.709045 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392563e-d84b-49a4-a2f4-fcce431583d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:57.709912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392563e-d84b-49a4-a2f4-fcce431583d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:57.710260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392563e-d84b-49a4-a2f4-fcce431583d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:57.710602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392563e-d84b-49a4-a2f4-fcce431583d1 admin admin] PATH_INFO: `/v3/projects/7871ffa539134a17a32794b561e1ec29` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:57.712475 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392563e-d84b-49a4-a2f4-fcce431583d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7871ffa539134a17a32794b561e1ec29)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:57.714567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392563e-d84b-49a4-a2f4-fcce431583d1 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:57.714832 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43700ab-4289-4a56-ad7b-e99f274145aa admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:57.721608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03830484-dd8e-4372-8ba3-446ea7416a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:57.722398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03830484-dd8e-4372-8ba3-446ea7416a6e admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:57.723258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03830484-dd8e-4372-8ba3-446ea7416a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:57.723706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03830484-dd8e-4372-8ba3-446ea7416a6e admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:57.724430 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03830484-dd8e-4372-8ba3-446ea7416a6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:57.727055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03830484-dd8e-4372-8ba3-446ea7416a6e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:57.756334 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 933/1886] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:57 2022] POST /identity/v3/users => generated 299 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:57.756334 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03830484-dd8e-4372-8ba3-446ea7416a6e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:57.758200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-03830484-dd8e-4372-8ba3-446ea7416a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:57.775378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b392563e-d84b-49a4-a2f4-fcce431583d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7871ffa539134a17a32794b561e1ec29', 'request_id': 'req-b392563e-d84b-49a4-a2f4-fcce431583d1'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:57.777113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:57.782115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b392563e-d84b-49a4-a2f4-fcce431583d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7871ffa539134a17a32794b561e1ec29', 'request_id': 'req-b392563e-d84b-49a4-a2f4-fcce431583d1'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:57.790640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b392563e-d84b-49a4-a2f4-fcce431583d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7871ffa539134a17a32794b561e1ec29', 'request_id': 'req-b392563e-d84b-49a4-a2f4-fcce431583d1'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:57.804855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b392563e-d84b-49a4-a2f4-fcce431583d1 admin admin] The token cache is being invalidate because project 7871ffa539134a17a32794b561e1ec29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:57.805672 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b392563e-d84b-49a4-a2f4-fcce431583d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:57.805976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b392563e-d84b-49a4-a2f4-fcce431583d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:57.811536 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 954/1887] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:57 2022] DELETE /identity/v3/projects/7871ffa539134a17a32794b561e1ec29 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:57.812632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392563e-d84b-49a4-a2f4-fcce431583d1 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:57.814176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b392563e-d84b-49a4-a2f4-fcce431583d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:57.833482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03830484-dd8e-4372-8ba3-446ea7416a6e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:57.850417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:57.889906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392563e-d84b-49a4-a2f4-fcce431583d1 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:57.897343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86219e96-452a-4d42-8f6b-b3dfee9fdd3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:57.897501 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fb1a2-32b1-4374-9e2a-389ac93e6646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:57.898259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86219e96-452a-4d42-8f6b-b3dfee9fdd3d service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:57.898402 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fb1a2-32b1-4374-9e2a-389ac93e6646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:57.898577 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fb1a2-32b1-4374-9e2a-389ac93e6646 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:57.898775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86219e96-452a-4d42-8f6b-b3dfee9fdd3d service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:57.898881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fb1a2-32b1-4374-9e2a-389ac93e6646 admin admin] PATH_INFO: `/v3/projects/dce9de7e98844951b9b4b420512540db` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:57.899163 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86219e96-452a-4d42-8f6b-b3dfee9fdd3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:57.900272 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291fb1a2-32b1-4374-9e2a-389ac93e6646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce9de7e98844951b9b4b420512540db)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:57.901440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-86219e96-452a-4d42-8f6b-b3dfee9fdd3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:57.903120 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291fb1a2-32b1-4374-9e2a-389ac93e6646 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:57.967796 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-291fb1a2-32b1-4374-9e2a-389ac93e6646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce9de7e98844951b9b4b420512540db', 'request_id': 'req-291fb1a2-32b1-4374-9e2a-389ac93e6646'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:57.974026 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-291fb1a2-32b1-4374-9e2a-389ac93e6646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce9de7e98844951b9b4b420512540db', 'request_id': 'req-291fb1a2-32b1-4374-9e2a-389ac93e6646'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:57.985309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-291fb1a2-32b1-4374-9e2a-389ac93e6646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce9de7e98844951b9b4b420512540db', 'request_id': 'req-291fb1a2-32b1-4374-9e2a-389ac93e6646'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:58.000212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-291fb1a2-32b1-4374-9e2a-389ac93e6646 admin admin] The token cache is being invalidate because project dce9de7e98844951b9b4b420512540db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:58.000611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-291fb1a2-32b1-4374-9e2a-389ac93e6646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:58.001578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-291fb1a2-32b1-4374-9e2a-389ac93e6646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:58.002320 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86219e96-452a-4d42-8f6b-b3dfee9fdd3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:58.003529 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 955/1888] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:57 2022] DELETE /identity/v3/projects/dce9de7e98844951b9b4b420512540db => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:58.004351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fb1a2-32b1-4374-9e2a-389ac93e6646 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:58.005353 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86219e96-452a-4d42-8f6b-b3dfee9fdd3d service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:58.005713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-291fb1a2-32b1-4374-9e2a-389ac93e6646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:58.016763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 934/1889] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:58.017946 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86219e96-452a-4d42-8f6b-b3dfee9fdd3d service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:58.019597 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-86219e96-452a-4d42-8f6b-b3dfee9fdd3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:58.035319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86219e96-452a-4d42-8f6b-b3dfee9fdd3d service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:58.116186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fb1a2-32b1-4374-9e2a-389ac93e6646 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:58.120930 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9999c-fb05-441d-8943-d7681ff005ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:58.121551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9999c-fb05-441d-8943-d7681ff005ab admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.121835 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9999c-fb05-441d-8943-d7681ff005ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.122110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9999c-fb05-441d-8943-d7681ff005ab admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.123615 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9999c-fb05-441d-8943-d7681ff005ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:58.125025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9999c-fb05-441d-8943-d7681ff005ab admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:58.129519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13da01-1da0-4a79-a81a-bbfe22e8728c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:58.130175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13da01-1da0-4a79-a81a-bbfe22e8728c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.130460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13da01-1da0-4a79-a81a-bbfe22e8728c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.130737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13da01-1da0-4a79-a81a-bbfe22e8728c admin admin] PATH_INFO: `/v3/users/118fa44ee22b43d4af2c8bc2e6299eb8` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.131998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13da01-1da0-4a79-a81a-bbfe22e8728c admin admin] RBAC: Authorizing `identity:delete_user(user_id=118fa44ee22b43d4af2c8bc2e6299eb8)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:58.133971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13da01-1da0-4a79-a81a-bbfe22e8728c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:58.135600 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 956/1890] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:58.136924 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8663d1f-f8b8-4783-a652-603fd1b72677 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.137306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8663d1f-f8b8-4783-a652-603fd1b72677 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.137564 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8663d1f-f8b8-4783-a652-603fd1b72677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.156314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:58.158145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d8663d1f-f8b8-4783-a652-603fd1b72677 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:58.159453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d8663d1f-f8b8-4783-a652-603fd1b72677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:58.169489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 957/1891] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:58.170513 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8663d1f-f8b8-4783-a652-603fd1b72677 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:58.172019 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d8663d1f-f8b8-4783-a652-603fd1b72677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:58.184402 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3c13da01-1da0-4a79-a81a-bbfe22e8728c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '118fa44ee22b43d4af2c8bc2e6299eb8'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:58.192789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3c13da01-1da0-4a79-a81a-bbfe22e8728c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '118fa44ee22b43d4af2c8bc2e6299eb8'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:58.205518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3c13da01-1da0-4a79-a81a-bbfe22e8728c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '118fa44ee22b43d4af2c8bc2e6299eb8'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:58.212424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3c13da01-1da0-4a79-a81a-bbfe22e8728c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '118fa44ee22b43d4af2c8bc2e6299eb8'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:58.213131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3c13da01-1da0-4a79-a81a-bbfe22e8728c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '118fa44ee22b43d4af2c8bc2e6299eb8'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:58.219397 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 935/1892] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:58 2022] DELETE /identity/v3/users/118fa44ee22b43d4af2c8bc2e6299eb8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:58.220211 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13da01-1da0-4a79-a81a-bbfe22e8728c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:58.221606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-3c13da01-1da0-4a79-a81a-bbfe22e8728c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:58.239313 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8663d1f-f8b8-4783-a652-603fd1b72677 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:58.243883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:58.314263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb64ed-b74b-499f-bbab-ceee186dabdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:58.314906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb64ed-b74b-499f-bbab-ceee186dabdb service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.315190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb64ed-b74b-499f-bbab-ceee186dabdb service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.315464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb64ed-b74b-499f-bbab-ceee186dabdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.317650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a8cb64ed-b74b-499f-bbab-ceee186dabdb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:58.319427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13da01-1da0-4a79-a81a-bbfe22e8728c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:58.325885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccb50f-e351-44c4-9045-fa6886ee3b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:58.326607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccb50f-e351-44c4-9045-fa6886ee3b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.326911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccb50f-e351-44c4-9045-fa6886ee3b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.327395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccb50f-e351-44c4-9045-fa6886ee3b9b admin admin] PATH_INFO: `/v3/projects/e0f1136e03b34398a55823dd3a603257/users/5631bdc254cf4df1ae34a1cdb900461c/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.346160 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ccb50f-e351-44c4-9045-fa6886ee3b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f1136e03b34398a55823dd3a603257, user_id=5631bdc254cf4df1ae34a1cdb900461c, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:58.349138 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ccb50f-e351-44c4-9045-fa6886ee3b9b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:58.368707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 936/1893] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:58 2022] PUT /identity/v3/projects/e0f1136e03b34398a55823dd3a603257/users/5631bdc254cf4df1ae34a1cdb900461c/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:58.370171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45212b71-75a3-4041-be91-551099217c63 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.370629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45212b71-75a3-4041-be91-551099217c63 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.371052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45212b71-75a3-4041-be91-551099217c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.385606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb64ed-b74b-499f-bbab-ceee186dabdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:58.387079 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb64ed-b74b-499f-bbab-ceee186dabdb service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:58.396216 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-45212b71-75a3-4041-be91-551099217c63 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:58.397709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-45212b71-75a3-4041-be91-551099217c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:58.398522 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 958/1894] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:58.399374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb64ed-b74b-499f-bbab-ceee186dabdb service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:58.411359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb64ed-b74b-499f-bbab-ceee186dabdb service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:58.486065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 937/1895] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:58.491509 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45212b71-75a3-4041-be91-551099217c63 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:58.501842 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c202c35-718d-4471-b780-1b2001b99a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:58.502689 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c202c35-718d-4471-b780-1b2001b99a4d admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.503011 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c202c35-718d-4471-b780-1b2001b99a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.503505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c202c35-718d-4471-b780-1b2001b99a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.504010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45212b71-75a3-4041-be91-551099217c63 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:58.504658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c202c35-718d-4471-b780-1b2001b99a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:58.506675 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c202c35-718d-4471-b780-1b2001b99a4d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:58.511110 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b86c94-bee1-4f58-9d77-e79530f4b4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:58.511835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b86c94-bee1-4f58-9d77-e79530f4b4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.512210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b86c94-bee1-4f58-9d77-e79530f4b4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.512590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b86c94-bee1-4f58-9d77-e79530f4b4d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.513317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b86c94-bee1-4f58-9d77-e79530f4b4d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:50:58.514346 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b86c94-bee1-4f58-9d77-e79530f4b4d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:58.516694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b86c94-bee1-4f58-9d77-e79530f4b4d6 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:58.521278 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 959/1896] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:58.525043 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c202c35-718d-4471-b780-1b2001b99a4d admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:58.531473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 938/1897] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:50:58.535927 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b86c94-bee1-4f58-9d77-e79530f4b4d6 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:58.543797 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c202c35-718d-4471-b780-1b2001b99a4d admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:58.549916 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b86c94-bee1-4f58-9d77-e79530f4b4d6 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:58.551741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa82f3f-1ac9-4cb8-a182-54fab1f789d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:58.552846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa82f3f-1ac9-4cb8-a182-54fab1f789d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.553359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa82f3f-1ac9-4cb8-a182-54fab1f789d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.553821 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa82f3f-1ac9-4cb8-a182-54fab1f789d0 admin admin] PATH_INFO: `/v3/projects/e0f1136e03b34398a55823dd3a603257/users/5631bdc254cf4df1ae34a1cdb900461c/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.555889 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade1212-f8ab-4806-85ea-982790e2a323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:58.556422 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa82f3f-1ac9-4cb8-a182-54fab1f789d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f1136e03b34398a55823dd3a603257, user_id=5631bdc254cf4df1ae34a1cdb900461c, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:58.556669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade1212-f8ab-4806-85ea-982790e2a323 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.557121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade1212-f8ab-4806-85ea-982790e2a323 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.557441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade1212-f8ab-4806-85ea-982790e2a323 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.558281 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade1212-f8ab-4806-85ea-982790e2a323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:58.559486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa82f3f-1ac9-4cb8-a182-54fab1f789d0 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:58.562106 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade1212-f8ab-4806-85ea-982790e2a323 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:58.567556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:58.577081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 960/1898] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:58 2022] PUT /identity/v3/projects/e0f1136e03b34398a55823dd3a603257/users/5631bdc254cf4df1ae34a1cdb900461c/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:58.594476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406862a-2200-48d4-8909-d734968b1b70 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.594860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406862a-2200-48d4-8909-d734968b1b70 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.595228 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406862a-2200-48d4-8909-d734968b1b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.599537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 939/1899] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:58 2022] POST /identity/v3/projects => generated 389 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:58.603723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade1212-f8ab-4806-85ea-982790e2a323 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:58.649697 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-a406862a-2200-48d4-8909-d734968b1b70 None None] MFA Rules not processed for user `5631bdc254cf4df1ae34a1cdb900461c`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:58.653281 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a406862a-2200-48d4-8909-d734968b1b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:58.703501 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:58.711315 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade1212-f8ab-4806-85ea-982790e2a323 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:58.728030 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d29ea8-43e3-478c-84f3-52ab314fb8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:58.728743 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d29ea8-43e3-478c-84f3-52ab314fb8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.729145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d29ea8-43e3-478c-84f3-52ab314fb8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.729462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d29ea8-43e3-478c-84f3-52ab314fb8b6 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.730159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d29ea8-43e3-478c-84f3-52ab314fb8b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:58.733440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d29ea8-43e3-478c-84f3-52ab314fb8b6 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:58.765084 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 961/1900] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:50:58.776655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406862a-2200-48d4-8909-d734968b1b70 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:58.778834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a406862a-2200-48d4-8909-d734968b1b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:58.785463 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 940/1901] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:58 2022] POST /identity/v3/users => generated 295 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:58.787413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d29ea8-43e3-478c-84f3-52ab314fb8b6 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:58.815339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d29ea8-43e3-478c-84f3-52ab314fb8b6 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:58.831830 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738affe8-c90f-4c6b-9fcd-74e19f0d5b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:58.832518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738affe8-c90f-4c6b-9fcd-74e19f0d5b82 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.832807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738affe8-c90f-4c6b-9fcd-74e19f0d5b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.833088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738affe8-c90f-4c6b-9fcd-74e19f0d5b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.833802 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738affe8-c90f-4c6b-9fcd-74e19f0d5b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:58.835689 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738affe8-c90f-4c6b-9fcd-74e19f0d5b82 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:58.862839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 941/1902] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:58.863801 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738affe8-c90f-4c6b-9fcd-74e19f0d5b82 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:58.883863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406862a-2200-48d4-8909-d734968b1b70 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:58.888691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230837ce-5f20-491a-8982-6a4673fb9e1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:58.889630 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230837ce-5f20-491a-8982-6a4673fb9e1f service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.889840 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738affe8-c90f-4c6b-9fcd-74e19f0d5b82 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:58.890069 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230837ce-5f20-491a-8982-6a4673fb9e1f service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.890357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230837ce-5f20-491a-8982-6a4673fb9e1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.895177 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fe6bf-fb40-4981-9172-a59203de9219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:58.895824 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fe6bf-fb40-4981-9172-a59203de9219 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.896111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fe6bf-fb40-4981-9172-a59203de9219 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.896427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fe6bf-fb40-4981-9172-a59203de9219 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.897143 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5fe6bf-fb40-4981-9172-a59203de9219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:58.898779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5fe6bf-fb40-4981-9172-a59203de9219 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:58.902620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230837ce-5f20-491a-8982-6a4673fb9e1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:58.905374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230837ce-5f20-491a-8982-6a4673fb9e1f service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:58.912674 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 942/1903] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:58.922320 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fe6bf-fb40-4981-9172-a59203de9219 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:58.941886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 962/1904] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:58.943381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fe6bf-fb40-4981-9172-a59203de9219 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:58.960866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a2366-b122-42ee-a79b-13d674a6d3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:58.961577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a2366-b122-42ee-a79b-13d674a6d3dd admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:58.961866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a2366-b122-42ee-a79b-13d674a6d3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:58.962155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a2366-b122-42ee-a79b-13d674a6d3dd admin admin] PATH_INFO: `/v3/projects/5fb46b96bd3149f8b86e023b6cf34dc3/users/d63f39e982674d8a964d60eb1aca9847/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:58.997002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a2366-b122-42ee-a79b-13d674a6d3dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fb46b96bd3149f8b86e023b6cf34dc3, user_id=d63f39e982674d8a964d60eb1aca9847, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:59.000032 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a2366-b122-42ee-a79b-13d674a6d3dd admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:59.038582 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 943/1905] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:58 2022] PUT /identity/v3/projects/5fb46b96bd3149f8b86e023b6cf34dc3/users/d63f39e982674d8a964d60eb1aca9847/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:50:59.054031 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230837ce-5f20-491a-8982-6a4673fb9e1f service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:59.058234 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b22bab-621c-4a9c-8414-4c1d4aaafd56 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:59.058685 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b22bab-621c-4a9c-8414-4c1d4aaafd56 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:59.059527 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-230837ce-5f20-491a-8982-6a4673fb9e1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:59.061174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b22bab-621c-4a9c-8414-4c1d4aaafd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:59.096533 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230837ce-5f20-491a-8982-6a4673fb9e1f service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:59.124707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-37b22bab-621c-4a9c-8414-4c1d4aaafd56 None None] MFA Rules not processed for user `d63f39e982674d8a964d60eb1aca9847`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:59.127042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-37b22bab-621c-4a9c-8414-4c1d4aaafd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:59.196571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d87ee-46d7-4ebc-b8fc-7c9d35586c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:59.197256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d87ee-46d7-4ebc-b8fc-7c9d35586c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:59.197587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d87ee-46d7-4ebc-b8fc-7c9d35586c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:59.197872 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d87ee-46d7-4ebc-b8fc-7c9d35586c86 admin admin] PATH_INFO: `/v3/projects/d10cd3d5f89e4ad485b87efcba3de95d` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:59.199298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d87ee-46d7-4ebc-b8fc-7c9d35586c86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d10cd3d5f89e4ad485b87efcba3de95d)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:59.201887 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d87ee-46d7-4ebc-b8fc-7c9d35586c86 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:59.288110 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 944/1906] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:59.292860 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b22bab-621c-4a9c-8414-4c1d4aaafd56 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:59.360908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e62d87ee-46d7-4ebc-b8fc-7c9d35586c86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd10cd3d5f89e4ad485b87efcba3de95d', 'request_id': 'req-e62d87ee-46d7-4ebc-b8fc-7c9d35586c86'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:59.369369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b22bab-621c-4a9c-8414-4c1d4aaafd56 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:59.393109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e62d87ee-46d7-4ebc-b8fc-7c9d35586c86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd10cd3d5f89e4ad485b87efcba3de95d', 'request_id': 'req-e62d87ee-46d7-4ebc-b8fc-7c9d35586c86'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:59.415496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e62d87ee-46d7-4ebc-b8fc-7c9d35586c86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd10cd3d5f89e4ad485b87efcba3de95d', 'request_id': 'req-e62d87ee-46d7-4ebc-b8fc-7c9d35586c86'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:59.425489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:59.437251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e62d87ee-46d7-4ebc-b8fc-7c9d35586c86 admin admin] The token cache is being invalidate because project d10cd3d5f89e4ad485b87efcba3de95d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:50:59.437639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e62d87ee-46d7-4ebc-b8fc-7c9d35586c86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:59.439890 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e62d87ee-46d7-4ebc-b8fc-7c9d35586c86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:50:59.445416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 963/1907] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:50:59 2022] DELETE /identity/v3/projects/d10cd3d5f89e4ad485b87efcba3de95d => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:59.450401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc1e81-cef4-43fa-938c-355c442e8fae None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:59.450695 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc1e81-cef4-43fa-938c-355c442e8fae None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:59.451065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc1e81-cef4-43fa-938c-355c442e8fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:59.478251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:50:59.509595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-0cbc1e81-cef4-43fa-938c-355c442e8fae None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:59.513170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0cbc1e81-cef4-43fa-938c-355c442e8fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:59.557797 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9823e-f654-47de-9919-fa52c989fb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:59.558509 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9823e-f654-47de-9919-fa52c989fb3f admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:59.558795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9823e-f654-47de-9919-fa52c989fb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:59.559068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9823e-f654-47de-9919-fa52c989fb3f admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:59.561531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9823e-f654-47de-9919-fa52c989fb3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:59.565236 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9823e-f654-47de-9919-fa52c989fb3f admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:59.627519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 945/1908] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:50:59 2022] POST /identity/v3/projects => generated 387 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:59.633613 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3d8b9-f2d2-4e13-8987-864f1ae784f3 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:59.633905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3d8b9-f2d2-4e13-8987-864f1ae784f3 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:59.634184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3d8b9-f2d2-4e13-8987-864f1ae784f3 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:59.635515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 946/1909] 10.222.0.69 () {54 vars in 921 bytes} [Wed Aug 3 18:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:50:59.636196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3d8b9-f2d2-4e13-8987-864f1ae784f3 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:59.642034 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-78a3d8b9-f2d2-4e13-8987-864f1ae784f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:59.657019 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 964/1910] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:59.658470 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468a7e7-68cc-40df-a85f-b55db92fd74b None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:59.658813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468a7e7-68cc-40df-a85f-b55db92fd74b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:59.659133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468a7e7-68cc-40df-a85f-b55db92fd74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:59.679562 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-8468a7e7-68cc-40df-a85f-b55db92fd74b None None] MFA Rules not processed for user `850a50cfe4394febbafb0908b08888b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:59.680833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8468a7e7-68cc-40df-a85f-b55db92fd74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:59.747207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3d8b9-f2d2-4e13-8987-864f1ae784f3 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:59.753953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94935ba7-71d7-427e-9eb6-69363f0800dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:59.754929 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94935ba7-71d7-427e-9eb6-69363f0800dc admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:59.755283 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94935ba7-71d7-427e-9eb6-69363f0800dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:59.755630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94935ba7-71d7-427e-9eb6-69363f0800dc admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:59.756541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94935ba7-71d7-427e-9eb6-69363f0800dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:59.758880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94935ba7-71d7-427e-9eb6-69363f0800dc admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:59.795324 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 947/1911] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:50:59 2022] POST /identity/v3/users => generated 300 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:50:59.797500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 965/1912] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Aug 3 18:50:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:59.798197 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b0ed0-14f8-4e31-8574-cd771a2f7827 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:59.798692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b0ed0-14f8-4e31-8574-cd771a2f7827 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:59.798781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468a7e7-68cc-40df-a85f-b55db92fd74b None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:59.799074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b0ed0-14f8-4e31-8574-cd771a2f7827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:59.821027 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468a7e7-68cc-40df-a85f-b55db92fd74b None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:59.829439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d814209-f261-4b8d-9e88-934bbcf84dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:59.830095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d814209-f261-4b8d-9e88-934bbcf84dbb admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:59.830432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d814209-f261-4b8d-9e88-934bbcf84dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:59.830779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d814209-f261-4b8d-9e88-934bbcf84dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:59.831551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d814209-f261-4b8d-9e88-934bbcf84dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:59.833296 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d814209-f261-4b8d-9e88-934bbcf84dbb admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:59.836873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-666b0ed0-14f8-4e31-8574-cd771a2f7827 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:50:59.838836 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-666b0ed0-14f8-4e31-8574-cd771a2f7827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:59.846397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 966/1913] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:50:59.847456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d814209-f261-4b8d-9e88-934bbcf84dbb admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:59.851086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7d814209-f261-4b8d-9e88-934bbcf84dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:59.853647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 948/1914] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:50:59.854608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b0ed0-14f8-4e31-8574-cd771a2f7827 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:59.881483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d814209-f261-4b8d-9e88-934bbcf84dbb admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:59.885925 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b0ed0-14f8-4e31-8574-cd771a2f7827 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:50:59.890624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68573e6b-ff84-4d9a-b2fc-6c2e511ac512 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:59.891538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68573e6b-ff84-4d9a-b2fc-6c2e511ac512 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:59.891828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68573e6b-ff84-4d9a-b2fc-6c2e511ac512 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:59.892241 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68573e6b-ff84-4d9a-b2fc-6c2e511ac512 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:59.896936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0d229-6d3b-4ac6-ab82-dc3d63841859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:50:59.897757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0d229-6d3b-4ac6-ab82-dc3d63841859 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:50:59.898074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0d229-6d3b-4ac6-ab82-dc3d63841859 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:50:59.898527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0d229-6d3b-4ac6-ab82-dc3d63841859 admin admin] PATH_INFO: `/v3/projects/692c4ef4817a40ada6c1c312af107ed2/users/d7544836f55a49d0a6a966752898be76/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:50:59.927987 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68573e6b-ff84-4d9a-b2fc-6c2e511ac512 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:59.929835 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68573e6b-ff84-4d9a-b2fc-6c2e511ac512 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:59.940247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0d229-6d3b-4ac6-ab82-dc3d63841859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=692c4ef4817a40ada6c1c312af107ed2, user_id=d7544836f55a49d0a6a966752898be76, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:50:59.943153 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0d229-6d3b-4ac6-ab82-dc3d63841859 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:50:59.960207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 949/1915] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:50:59 2022] PUT /identity/v3/projects/692c4ef4817a40ada6c1c312af107ed2/users/d7544836f55a49d0a6a966752898be76/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:50:59.963346 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0d229-6d3b-4ac6-ab82-dc3d63841859 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:59.967328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 967/1916] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:50:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:50:59.968223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68573e6b-ff84-4d9a-b2fc-6c2e511ac512 service nova] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:50:59.970937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-68573e6b-ff84-4d9a-b2fc-6c2e511ac512 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:50:59.987412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68573e6b-ff84-4d9a-b2fc-6c2e511ac512 service nova] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:00.083504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a337146-3511-41d9-9515-4428c768c03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:00.084154 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a337146-3511-41d9-9515-4428c768c03d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:00.084597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a337146-3511-41d9-9515-4428c768c03d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:00.084886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a337146-3511-41d9-9515-4428c768c03d admin admin] PATH_INFO: `/v3/users/49f2a958c5394fc5877fc468f0916d34` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:00.086085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a337146-3511-41d9-9515-4428c768c03d admin admin] RBAC: Authorizing `identity:delete_user(user_id=49f2a958c5394fc5877fc468f0916d34)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:00.088016 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a337146-3511-41d9-9515-4428c768c03d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:00.102679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:00.112735 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0d229-6d3b-4ac6-ab82-dc3d63841859 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:00.118662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ae4b7-9177-4373-8864-e062e1f0afe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:00.119537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ae4b7-9177-4373-8864-e062e1f0afe4 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:00.120096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ae4b7-9177-4373-8864-e062e1f0afe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:00.120783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ae4b7-9177-4373-8864-e062e1f0afe4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:00.121304 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ae4b7-9177-4373-8864-e062e1f0afe4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:51:00.121707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ae4b7-9177-4373-8864-e062e1f0afe4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:00.124863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ae4b7-9177-4373-8864-e062e1f0afe4 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:00.131076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:00.152911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 950/1917] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:51:00.153827 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ae4b7-9177-4373-8864-e062e1f0afe4 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:00.164715 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4a337146-3511-41d9-9515-4428c768c03d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49f2a958c5394fc5877fc468f0916d34'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:00.178170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ae4b7-9177-4373-8864-e062e1f0afe4 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:00.187426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4a337146-3511-41d9-9515-4428c768c03d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49f2a958c5394fc5877fc468f0916d34'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:00.190352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cea3ba-6964-4ccc-8c2b-35cafb0cacd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:00.191179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cea3ba-6964-4ccc-8c2b-35cafb0cacd8 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:00.191582 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cea3ba-6964-4ccc-8c2b-35cafb0cacd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:00.191950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cea3ba-6964-4ccc-8c2b-35cafb0cacd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:00.192759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cea3ba-6964-4ccc-8c2b-35cafb0cacd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:00.194490 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cea3ba-6964-4ccc-8c2b-35cafb0cacd8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:00.204807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4a337146-3511-41d9-9515-4428c768c03d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49f2a958c5394fc5877fc468f0916d34'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:00.213896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 951/1918] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:00.215163 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cea3ba-6964-4ccc-8c2b-35cafb0cacd8 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:00.215861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4a337146-3511-41d9-9515-4428c768c03d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49f2a958c5394fc5877fc468f0916d34'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:00.218024 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4a337146-3511-41d9-9515-4428c768c03d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49f2a958c5394fc5877fc468f0916d34'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:00.229820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cea3ba-6964-4ccc-8c2b-35cafb0cacd8 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:00.230983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 968/1919] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:50:59 2022] DELETE /identity/v3/users/49f2a958c5394fc5877fc468f0916d34 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:00.232051 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a337146-3511-41d9-9515-4428c768c03d admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:00.234181 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4a337146-3511-41d9-9515-4428c768c03d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:00.319424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44706f-baeb-4ba0-8394-ef3cb805a36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:00.320061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44706f-baeb-4ba0-8394-ef3cb805a36a admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:00.320378 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44706f-baeb-4ba0-8394-ef3cb805a36a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:00.320701 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44706f-baeb-4ba0-8394-ef3cb805a36a admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:00.321661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44706f-baeb-4ba0-8394-ef3cb805a36a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:00.324175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44706f-baeb-4ba0-8394-ef3cb805a36a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:00.331067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a337146-3511-41d9-9515-4428c768c03d admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:00.336226 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857675dd-a739-467b-9144-ca6834aefd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:00.337291 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857675dd-a739-467b-9144-ca6834aefd4f admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:00.337550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857675dd-a739-467b-9144-ca6834aefd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:00.337829 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857675dd-a739-467b-9144-ca6834aefd4f admin admin] PATH_INFO: `/v3/projects/692c4ef4817a40ada6c1c312af107ed2/users/d7544836f55a49d0a6a966752898be76/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:00.340312 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857675dd-a739-467b-9144-ca6834aefd4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=692c4ef4817a40ada6c1c312af107ed2, user_id=d7544836f55a49d0a6a966752898be76, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:00.353080 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857675dd-a739-467b-9144-ca6834aefd4f admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:00.366629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 969/1920] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:00 2022] PUT /identity/v3/projects/692c4ef4817a40ada6c1c312af107ed2/users/d7544836f55a49d0a6a966752898be76/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:00.368935 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5e128-a0af-4e4c-89be-d89feb02d8c8 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:00.369432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5e128-a0af-4e4c-89be-d89feb02d8c8 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:00.369925 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5e128-a0af-4e4c-89be-d89feb02d8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:00.391443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 952/1921] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:00 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:00.393268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52ba73-fa9c-4f81-9d33-1f810e1d6ff0 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:00.393561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52ba73-fa9c-4f81-9d33-1f810e1d6ff0 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:00.393836 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52ba73-fa9c-4f81-9d33-1f810e1d6ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:00.405601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-26f5e128-a0af-4e4c-89be-d89feb02d8c8 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:00.407444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-26f5e128-a0af-4e4c-89be-d89feb02d8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:00.440408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ce52ba73-fa9c-4f81-9d33-1f810e1d6ff0 None None] MFA Rules not processed for user `d7544836f55a49d0a6a966752898be76`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:00.440408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ce52ba73-fa9c-4f81-9d33-1f810e1d6ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:00.489524 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 970/1922] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:00.493020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5e128-a0af-4e4c-89be-d89feb02d8c8 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:00.493020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-26f5e128-a0af-4e4c-89be-d89feb02d8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:00.507507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5e128-a0af-4e4c-89be-d89feb02d8c8 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:00.512555 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531fd06-76d3-44e7-b02d-1fa6ce444395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:00.513190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531fd06-76d3-44e7-b02d-1fa6ce444395 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:00.513537 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531fd06-76d3-44e7-b02d-1fa6ce444395 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:00.513850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531fd06-76d3-44e7-b02d-1fa6ce444395 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:00.516075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531fd06-76d3-44e7-b02d-1fa6ce444395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:00.517028 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531fd06-76d3-44e7-b02d-1fa6ce444395 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:00.537727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 953/1923] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:00.541310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52ba73-fa9c-4f81-9d33-1f810e1d6ff0 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:00.546940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ce52ba73-fa9c-4f81-9d33-1f810e1d6ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:00.555579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 971/1924] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:00 2022] POST /identity/v3/users => generated 294 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:00.556893 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531fd06-76d3-44e7-b02d-1fa6ce444395 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:00.562056 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d531fd06-76d3-44e7-b02d-1fa6ce444395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:00.575075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:00.612764 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52ba73-fa9c-4f81-9d33-1f810e1d6ff0 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:00.616494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627cf10e-72df-4da8-9195-32cfc493a316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:00.617011 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cf10e-72df-4da8-9195-32cfc493a316 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:00.617289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cf10e-72df-4da8-9195-32cfc493a316 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:00.617562 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cf10e-72df-4da8-9195-32cfc493a316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:00.627714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627cf10e-72df-4da8-9195-32cfc493a316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:00.629432 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627cf10e-72df-4da8-9195-32cfc493a316 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:00.630302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531fd06-76d3-44e7-b02d-1fa6ce444395 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:00.633537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:00.635206 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892dc1b-069e-4eed-bf22-0383363bd7ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:00.636017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892dc1b-069e-4eed-bf22-0383363bd7ba service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:00.636318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892dc1b-069e-4eed-bf22-0383363bd7ba service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:00.636601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892dc1b-069e-4eed-bf22-0383363bd7ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:00.640505 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 954/1925] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:00.641352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627cf10e-72df-4da8-9195-32cfc493a316 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:00.648052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892dc1b-069e-4eed-bf22-0383363bd7ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:00.649721 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892dc1b-069e-4eed-bf22-0383363bd7ba service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:00.652254 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627cf10e-72df-4da8-9195-32cfc493a316 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:00.657569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694bd75-e6e7-4c31-a2b3-5a248421e45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:00.658253 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694bd75-e6e7-4c31-a2b3-5a248421e45e admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:00.658602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694bd75-e6e7-4c31-a2b3-5a248421e45e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:00.659225 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694bd75-e6e7-4c31-a2b3-5a248421e45e admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:00.659792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694bd75-e6e7-4c31-a2b3-5a248421e45e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:00.661460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694bd75-e6e7-4c31-a2b3-5a248421e45e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:00.661910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 972/1926] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:00.673351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 955/1927] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:00.677962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892dc1b-069e-4eed-bf22-0383363bd7ba service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:00.701340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892dc1b-069e-4eed-bf22-0383363bd7ba service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:00.707691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdd574-7183-4902-a41c-ee6232fe2ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:00.708981 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdd574-7183-4902-a41c-ee6232fe2ade admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:00.709339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdd574-7183-4902-a41c-ee6232fe2ade admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:00.709752 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdd574-7183-4902-a41c-ee6232fe2ade admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:00.710718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdd574-7183-4902-a41c-ee6232fe2ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:00.712778 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdd574-7183-4902-a41c-ee6232fe2ade admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:00.730298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 973/1928] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:00.735356 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694bd75-e6e7-4c31-a2b3-5a248421e45e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:00.749781 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694bd75-e6e7-4c31-a2b3-5a248421e45e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:00.757240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7cd66-ae00-443a-bffe-ee0a3ecc05a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:00.758044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7cd66-ae00-443a-bffe-ee0a3ecc05a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:00.758451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7cd66-ae00-443a-bffe-ee0a3ecc05a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:00.758997 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7cd66-ae00-443a-bffe-ee0a3ecc05a7 admin admin] PATH_INFO: `/v3/projects/401d2ab88cf346cc86ecf524effbdb2d/users/57d8f562ca114aca8a6b0ff974449fa7/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:00.780227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7cd66-ae00-443a-bffe-ee0a3ecc05a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=401d2ab88cf346cc86ecf524effbdb2d, user_id=57d8f562ca114aca8a6b0ff974449fa7, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:00.783244 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7cd66-ae00-443a-bffe-ee0a3ecc05a7 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:00.815145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 956/1929] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:00 2022] PUT /identity/v3/projects/401d2ab88cf346cc86ecf524effbdb2d/users/57d8f562ca114aca8a6b0ff974449fa7/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:51:00.817720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec98eb9-42ed-4284-b798-6de3123cbe03 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:00.818189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec98eb9-42ed-4284-b798-6de3123cbe03 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:00.818272 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec98eb9-42ed-4284-b798-6de3123cbe03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:00.838615 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7cd66-ae00-443a-bffe-ee0a3ecc05a7 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:00.866928 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-dec98eb9-42ed-4284-b798-6de3123cbe03 None None] MFA Rules not processed for user `57d8f562ca114aca8a6b0ff974449fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:00.868272 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-dec98eb9-42ed-4284-b798-6de3123cbe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:00.930127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7cd66-ae00-443a-bffe-ee0a3ecc05a7 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:00.937624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2be275-330c-4be3-b338-3f024ddf6bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:00.938785 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2be275-330c-4be3-b338-3f024ddf6bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:00.939068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2be275-330c-4be3-b338-3f024ddf6bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:00.939346 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2be275-330c-4be3-b338-3f024ddf6bc2 admin admin] PATH_INFO: `/v3/users/d63f39e982674d8a964d60eb1aca9847` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:00.940832 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2be275-330c-4be3-b338-3f024ddf6bc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d63f39e982674d8a964d60eb1aca9847)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:00.943680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2be275-330c-4be3-b338-3f024ddf6bc2 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:00.972352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 974/1930] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:00.979190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec98eb9-42ed-4284-b798-6de3123cbe03 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:00.993447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec98eb9-42ed-4284-b798-6de3123cbe03 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:01.002427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf01b32-0a70-4ac7-96f8-47c091eb670c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:01.003079 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf01b32-0a70-4ac7-96f8-47c091eb670c admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:01.003348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf01b32-0a70-4ac7-96f8-47c091eb670c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:01.003626 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf01b32-0a70-4ac7-96f8-47c091eb670c admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:01.004604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf01b32-0a70-4ac7-96f8-47c091eb670c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:01.006538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf01b32-0a70-4ac7-96f8-47c091eb670c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:01.034306 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9a2be275-330c-4be3-b338-3f024ddf6bc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd63f39e982674d8a964d60eb1aca9847'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.039829 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 975/1931] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:00 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:01.040541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf01b32-0a70-4ac7-96f8-47c091eb670c admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:01.042138 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-fbf01b32-0a70-4ac7-96f8-47c091eb670c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:01.049169 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9a2be275-330c-4be3-b338-3f024ddf6bc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd63f39e982674d8a964d60eb1aca9847'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.079182 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9a2be275-330c-4be3-b338-3f024ddf6bc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd63f39e982674d8a964d60eb1aca9847'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.103741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9a2be275-330c-4be3-b338-3f024ddf6bc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd63f39e982674d8a964d60eb1aca9847'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.104941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9a2be275-330c-4be3-b338-3f024ddf6bc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd63f39e982674d8a964d60eb1aca9847'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.113965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 957/1932] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:00 2022] DELETE /identity/v3/users/d63f39e982674d8a964d60eb1aca9847 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:01.117041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2be275-330c-4be3-b338-3f024ddf6bc2 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:01.118788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9a2be275-330c-4be3-b338-3f024ddf6bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:01.128704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf01b32-0a70-4ac7-96f8-47c091eb670c admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:01.145930 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:01.196904 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:01.210310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:01.210570 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:01.210885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:01.211145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32 admin admin] PATH_INFO: `/v3/projects/1cfdb70f6c034cfbaf6805e2b33e3c71` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:01.217150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cfdb70f6c034cfbaf6805e2b33e3c71)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:01.219137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:01.224715 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2be275-330c-4be3-b338-3f024ddf6bc2 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:01.229550 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c819b-66fd-402d-bf32-3cfca02c9ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:01.230190 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c819b-66fd-402d-bf32-3cfca02c9ce4 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:01.230475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c819b-66fd-402d-bf32-3cfca02c9ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:01.230757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c819b-66fd-402d-bf32-3cfca02c9ce4 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:01.231431 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c819b-66fd-402d-bf32-3cfca02c9ce4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:01.234313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c819b-66fd-402d-bf32-3cfca02c9ce4 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:01.260303 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 958/1933] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:01 2022] POST /identity/v3/users => generated 300 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:01.261860 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece394f0-c5da-457d-a563-9704e9e996b1 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:01.262152 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece394f0-c5da-457d-a563-9704e9e996b1 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:01.262429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece394f0-c5da-457d-a563-9704e9e996b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:01.283712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ece394f0-c5da-457d-a563-9704e9e996b1 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:01.284968 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ece394f0-c5da-457d-a563-9704e9e996b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:01.288752 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cfdb70f6c034cfbaf6805e2b33e3c71', 'request_id': 'req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.295235 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cfdb70f6c034cfbaf6805e2b33e3c71', 'request_id': 'req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.303926 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cfdb70f6c034cfbaf6805e2b33e3c71', 'request_id': 'req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.315534 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32 admin admin] The token cache is being invalidate because project 1cfdb70f6c034cfbaf6805e2b33e3c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:01.315826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.316539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.320905 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 976/1934] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:01 2022] DELETE /identity/v3/projects/1cfdb70f6c034cfbaf6805e2b33e3c71 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:01.320905 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:01.320905 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:01.353395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 959/1935] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:01.354263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece394f0-c5da-457d-a563-9704e9e996b1 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:01.355533 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ece394f0-c5da-457d-a563-9704e9e996b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:01.367470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece394f0-c5da-457d-a563-9704e9e996b1 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:01.372411 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434499e-93df-4b87-baa4-2b6e81f8ed7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:01.373094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434499e-93df-4b87-baa4-2b6e81f8ed7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:01.373553 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434499e-93df-4b87-baa4-2b6e81f8ed7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:01.373835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434499e-93df-4b87-baa4-2b6e81f8ed7a admin admin] PATH_INFO: `/v3/users/5631bdc254cf4df1ae34a1cdb900461c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:01.375184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434499e-93df-4b87-baa4-2b6e81f8ed7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5631bdc254cf4df1ae34a1cdb900461c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:01.377341 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434499e-93df-4b87-baa4-2b6e81f8ed7a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:01.400799 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb337ad9-5e69-48d4-ad97-f8bb71a88c32 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:01.407261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681f52b-b5fe-47ca-b96b-0d6247e7de57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:01.408178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681f52b-b5fe-47ca-b96b-0d6247e7de57 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:01.408673 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681f52b-b5fe-47ca-b96b-0d6247e7de57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:01.409128 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681f52b-b5fe-47ca-b96b-0d6247e7de57 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:01.410160 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681f52b-b5fe-47ca-b96b-0d6247e7de57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:01.412144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681f52b-b5fe-47ca-b96b-0d6247e7de57 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:01.427368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 977/1936] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:01.427368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681f52b-b5fe-47ca-b96b-0d6247e7de57 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:01.430347 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e681f52b-b5fe-47ca-b96b-0d6247e7de57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:01.448449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-0434499e-93df-4b87-baa4-2b6e81f8ed7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5631bdc254cf4df1ae34a1cdb900461c'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.460797 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-0434499e-93df-4b87-baa4-2b6e81f8ed7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5631bdc254cf4df1ae34a1cdb900461c'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.485690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-0434499e-93df-4b87-baa4-2b6e81f8ed7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5631bdc254cf4df1ae34a1cdb900461c'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.491436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:01.495717 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-0434499e-93df-4b87-baa4-2b6e81f8ed7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5631bdc254cf4df1ae34a1cdb900461c'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.496558 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-0434499e-93df-4b87-baa4-2b6e81f8ed7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5631bdc254cf4df1ae34a1cdb900461c'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.512338 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 960/1937] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:01 2022] DELETE /identity/v3/users/5631bdc254cf4df1ae34a1cdb900461c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:01.515407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434499e-93df-4b87-baa4-2b6e81f8ed7a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:01.517057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681f52b-b5fe-47ca-b96b-0d6247e7de57 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:01.519908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-0434499e-93df-4b87-baa4-2b6e81f8ed7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:01.606915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0ae0c-ada2-46b3-9864-411c466184f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:01.608331 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0ae0c-ada2-46b3-9864-411c466184f1 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:01.609605 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0ae0c-ada2-46b3-9864-411c466184f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:01.609973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0ae0c-ada2-46b3-9864-411c466184f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:01.612782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-eda0ae0c-ada2-46b3-9864-411c466184f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:01.646418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:01.706421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434499e-93df-4b87-baa4-2b6e81f8ed7a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:01.723784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75be21-27a7-4328-a168-516064bfeb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:01.724848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75be21-27a7-4328-a168-516064bfeb2f admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:01.725147 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75be21-27a7-4328-a168-516064bfeb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:01.725433 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75be21-27a7-4328-a168-516064bfeb2f admin admin] PATH_INFO: `/v3/projects/8c2191c9560d4a8b8275ab95837731c2/users/63ab4e2c8a844ed199a512862693dd5a/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:01.735351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0ae0c-ada2-46b3-9864-411c466184f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:01.739130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0ae0c-ada2-46b3-9864-411c466184f1 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:01.750912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 978/1938] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:01.752158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0ae0c-ada2-46b3-9864-411c466184f1 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:01.754667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-eda0ae0c-ada2-46b3-9864-411c466184f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:01.761137 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75be21-27a7-4328-a168-516064bfeb2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2191c9560d4a8b8275ab95837731c2, user_id=63ab4e2c8a844ed199a512862693dd5a, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:01.767483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75be21-27a7-4328-a168-516064bfeb2f admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:01.770294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0ae0c-ada2-46b3-9864-411c466184f1 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:01.776832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad923547-0340-4fdc-9c17-a7abd1f1090e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:01.777970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad923547-0340-4fdc-9c17-a7abd1f1090e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:01.778435 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad923547-0340-4fdc-9c17-a7abd1f1090e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:01.779530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad923547-0340-4fdc-9c17-a7abd1f1090e admin admin] PATH_INFO: `/v3/users/f0e472160e67460aa8fc724ed271e4f6` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:01.781688 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad923547-0340-4fdc-9c17-a7abd1f1090e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0e472160e67460aa8fc724ed271e4f6)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:01.784087 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad923547-0340-4fdc-9c17-a7abd1f1090e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:01.801073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 961/1939] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:01 2022] PUT /identity/v3/projects/8c2191c9560d4a8b8275ab95837731c2/users/63ab4e2c8a844ed199a512862693dd5a/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:01.802901 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75be21-27a7-4328-a168-516064bfeb2f admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:01.869505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ad923547-0340-4fdc-9c17-a7abd1f1090e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0e472160e67460aa8fc724ed271e4f6'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.884943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ad923547-0340-4fdc-9c17-a7abd1f1090e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0e472160e67460aa8fc724ed271e4f6'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.903797 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ad923547-0340-4fdc-9c17-a7abd1f1090e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0e472160e67460aa8fc724ed271e4f6'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.916334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ad923547-0340-4fdc-9c17-a7abd1f1090e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0e472160e67460aa8fc724ed271e4f6'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.916529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75be21-27a7-4328-a168-516064bfeb2f admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:01.917354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ad923547-0340-4fdc-9c17-a7abd1f1090e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0e472160e67460aa8fc724ed271e4f6'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:01.923306 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50848af7-b040-485a-98ac-2494a5b16bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:01.923704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50848af7-b040-485a-98ac-2494a5b16bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:01.924056 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50848af7-b040-485a-98ac-2494a5b16bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:01.924427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50848af7-b040-485a-98ac-2494a5b16bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:01.925104 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 979/1940] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:01 2022] DELETE /identity/v3/users/f0e472160e67460aa8fc724ed271e4f6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:01.925259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50848af7-b040-485a-98ac-2494a5b16bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:01.926952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50848af7-b040-485a-98ac-2494a5b16bc5 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:01.932356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff9f26-3a01-4424-a0bb-c6865f3d83a7 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:01.932356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff9f26-3a01-4424-a0bb-c6865f3d83a7 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:01.932356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff9f26-3a01-4424-a0bb-c6865f3d83a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:01.944893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 962/1941] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:01.949287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50848af7-b040-485a-98ac-2494a5b16bc5 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:01.951222 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-50848af7-b040-485a-98ac-2494a5b16bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:01.956196 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-77ff9f26-3a01-4424-a0bb-c6865f3d83a7 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:01.957583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-77ff9f26-3a01-4424-a0bb-c6865f3d83a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:01.965371 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50848af7-b040-485a-98ac-2494a5b16bc5 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:01.976710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:02.043209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71daf94c-df82-4a1d-aea4-31389872172a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:02.044127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daf94c-df82-4a1d-aea4-31389872172a admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:02.044469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daf94c-df82-4a1d-aea4-31389872172a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:02.045284 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daf94c-df82-4a1d-aea4-31389872172a admin admin] PATH_INFO: `/v3/projects/8c2191c9560d4a8b8275ab95837731c2/users/63ab4e2c8a844ed199a512862693dd5a/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:02.049601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 980/1942] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:51:02.049750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71daf94c-df82-4a1d-aea4-31389872172a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2191c9560d4a8b8275ab95837731c2, user_id=63ab4e2c8a844ed199a512862693dd5a, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:02.050313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71daf94c-df82-4a1d-aea4-31389872172a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:02.057081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff9f26-3a01-4424-a0bb-c6865f3d83a7 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:02.058594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-77ff9f26-3a01-4424-a0bb-c6865f3d83a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:02.062006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 963/1943] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:01 2022] PUT /identity/v3/projects/8c2191c9560d4a8b8275ab95837731c2/users/63ab4e2c8a844ed199a512862693dd5a/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:51:02.074011 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46ccbc-5902-4120-b8d5-51d46790ada0 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:02.074297 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46ccbc-5902-4120-b8d5-51d46790ada0 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:02.074637 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46ccbc-5902-4120-b8d5-51d46790ada0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:02.104128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-5c46ccbc-5902-4120-b8d5-51d46790ada0 None None] MFA Rules not processed for user `63ab4e2c8a844ed199a512862693dd5a`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:02.105424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-5c46ccbc-5902-4120-b8d5-51d46790ada0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:02.133195 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff9f26-3a01-4424-a0bb-c6865f3d83a7 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:02.177289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:02.212583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278deb1c-0377-434d-85aa-61ad8219b062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:02.213216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278deb1c-0377-434d-85aa-61ad8219b062 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:02.213492 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278deb1c-0377-434d-85aa-61ad8219b062 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:02.213765 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278deb1c-0377-434d-85aa-61ad8219b062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:02.254629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 964/1944] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:02.266579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46ccbc-5902-4120-b8d5-51d46790ada0 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:02.269060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-5c46ccbc-5902-4120-b8d5-51d46790ada0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:02.350375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278deb1c-0377-434d-85aa-61ad8219b062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:02.351781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278deb1c-0377-434d-85aa-61ad8219b062 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:02.379874 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 981/1945] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:02.380813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278deb1c-0377-434d-85aa-61ad8219b062 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:02.384316 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-278deb1c-0377-434d-85aa-61ad8219b062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:02.402618 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278deb1c-0377-434d-85aa-61ad8219b062 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:02.408688 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bd26a-a94b-42ab-9362-5748b7491d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:02.409360 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bd26a-a94b-42ab-9362-5748b7491d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:02.409650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bd26a-a94b-42ab-9362-5748b7491d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:02.409985 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bd26a-a94b-42ab-9362-5748b7491d2c admin admin] PATH_INFO: `/v3/projects/e0f1136e03b34398a55823dd3a603257` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:02.411429 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bd26a-a94b-42ab-9362-5748b7491d2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f1136e03b34398a55823dd3a603257)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:02.413850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bd26a-a94b-42ab-9362-5748b7491d2c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:02.423991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46ccbc-5902-4120-b8d5-51d46790ada0 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:02.429749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42faf8c-dee9-4927-9d74-532443f09d4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:02.430399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42faf8c-dee9-4927-9d74-532443f09d4f service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:02.430684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42faf8c-dee9-4927-9d74-532443f09d4f service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:02.431220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42faf8c-dee9-4927-9d74-532443f09d4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:02.445676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42faf8c-dee9-4927-9d74-532443f09d4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:02.448077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:02.448425 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42faf8c-dee9-4927-9d74-532443f09d4f service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:02.466384 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 965/1946] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:51:02.483925 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-2d3bd26a-a94b-42ab-9362-5748b7491d2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f1136e03b34398a55823dd3a603257', 'request_id': 'req-2d3bd26a-a94b-42ab-9362-5748b7491d2c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:02.490105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-2d3bd26a-a94b-42ab-9362-5748b7491d2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f1136e03b34398a55823dd3a603257', 'request_id': 'req-2d3bd26a-a94b-42ab-9362-5748b7491d2c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:02.506321 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-2d3bd26a-a94b-42ab-9362-5748b7491d2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f1136e03b34398a55823dd3a603257', 'request_id': 'req-2d3bd26a-a94b-42ab-9362-5748b7491d2c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:02.507547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42faf8c-dee9-4927-9d74-532443f09d4f service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:02.523424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42faf8c-dee9-4927-9d74-532443f09d4f service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:02.524798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-2d3bd26a-a94b-42ab-9362-5748b7491d2c admin admin] The token cache is being invalidate because project e0f1136e03b34398a55823dd3a603257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:02.525101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-2d3bd26a-a94b-42ab-9362-5748b7491d2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:02.525753 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-2d3bd26a-a94b-42ab-9362-5748b7491d2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:02.528968 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 982/1947] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:02 2022] DELETE /identity/v3/projects/e0f1136e03b34398a55823dd3a603257 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:02.535444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0c0ab-3433-469c-abbd-938cdc1d3bae None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:02.535751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0c0ab-3433-469c-abbd-938cdc1d3bae None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:02.536032 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0c0ab-3433-469c-abbd-938cdc1d3bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:02.560662 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-b5f0c0ab-3433-469c-abbd-938cdc1d3bae None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:02.562251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b5f0c0ab-3433-469c-abbd-938cdc1d3bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:02.605003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600006f2-8765-41ee-8a64-35770a66a52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:02.605692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600006f2-8765-41ee-8a64-35770a66a52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:02.606037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600006f2-8765-41ee-8a64-35770a66a52c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:02.606360 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600006f2-8765-41ee-8a64-35770a66a52c admin admin] PATH_INFO: `/v3/users/57d8f562ca114aca8a6b0ff974449fa7` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:02.610031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600006f2-8765-41ee-8a64-35770a66a52c admin admin] RBAC: Authorizing `identity:delete_user(user_id=57d8f562ca114aca8a6b0ff974449fa7)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:02.614094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600006f2-8765-41ee-8a64-35770a66a52c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:02.645020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 983/1948] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:02.652671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dfc58-c614-49e5-8556-8ad16881161a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:02.652963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dfc58-c614-49e5-8556-8ad16881161a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:02.653244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dfc58-c614-49e5-8556-8ad16881161a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:02.679246 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-6b1dfc58-c614-49e5-8556-8ad16881161a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:02.681674 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-6b1dfc58-c614-49e5-8556-8ad16881161a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:02.693626 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 984/1949] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:02.698004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-600006f2-8765-41ee-8a64-35770a66a52c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57d8f562ca114aca8a6b0ff974449fa7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:02.698586 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dfc58-c614-49e5-8556-8ad16881161a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:02.713288 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-600006f2-8765-41ee-8a64-35770a66a52c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57d8f562ca114aca8a6b0ff974449fa7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:02.715593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dfc58-c614-49e5-8556-8ad16881161a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:02.721718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f6dfd-8f2a-4231-8348-79e502b63ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:02.722766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f6dfd-8f2a-4231-8348-79e502b63ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:02.723148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f6dfd-8f2a-4231-8348-79e502b63ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:02.723526 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f6dfd-8f2a-4231-8348-79e502b63ac1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:02.724455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f6dfd-8f2a-4231-8348-79e502b63ac1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:51:02.724864 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f6dfd-8f2a-4231-8348-79e502b63ac1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:02.726703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f6dfd-8f2a-4231-8348-79e502b63ac1 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:02.733631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-600006f2-8765-41ee-8a64-35770a66a52c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57d8f562ca114aca8a6b0ff974449fa7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:02.742269 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-600006f2-8765-41ee-8a64-35770a66a52c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57d8f562ca114aca8a6b0ff974449fa7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:02.743014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-600006f2-8765-41ee-8a64-35770a66a52c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57d8f562ca114aca8a6b0ff974449fa7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:02.746160 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 985/1950] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:51:02.759501 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 966/1951] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:02 2022] DELETE /identity/v3/users/57d8f562ca114aca8a6b0ff974449fa7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:02.761045 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f6dfd-8f2a-4231-8348-79e502b63ac1 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:02.761553 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d3e5a-8d5d-4976-bf3d-1c1f25f53a9f None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:02.762015 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d3e5a-8d5d-4976-bf3d-1c1f25f53a9f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:02.762289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d3e5a-8d5d-4976-bf3d-1c1f25f53a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:02.766170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1c8f6dfd-8f2a-4231-8348-79e502b63ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:02.773688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:02.784781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f6dfd-8f2a-4231-8348-79e502b63ac1 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:02.790291 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-a01d3e5a-8d5d-4976-bf3d-1c1f25f53a9f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:02.793110 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a01d3e5a-8d5d-4976-bf3d-1c1f25f53a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:02.869319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a84c8-3602-497a-adea-4c9604d02fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:02.869964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a84c8-3602-497a-adea-4c9604d02fee admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:02.870532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a84c8-3602-497a-adea-4c9604d02fee admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:02.870532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a84c8-3602-497a-adea-4c9604d02fee admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:02.871506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a84c8-3602-497a-adea-4c9604d02fee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:02.874438 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a84c8-3602-497a-adea-4c9604d02fee admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:02.923523 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 986/1952] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:02 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:02.938249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a84c8-3602-497a-adea-4c9604d02fee admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:02.973564 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:02.975528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 967/1953] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:03.032343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d3e5a-8d5d-4976-bf3d-1c1f25f53a9f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:03.033958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a01d3e5a-8d5d-4976-bf3d-1c1f25f53a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:03.105458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a84c8-3602-497a-adea-4c9604d02fee admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:03.120640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a163ffd-d189-404f-a2c2-c22fb7521bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:03.122820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a163ffd-d189-404f-a2c2-c22fb7521bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:03.124709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a163ffd-d189-404f-a2c2-c22fb7521bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:03.124709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a163ffd-d189-404f-a2c2-c22fb7521bc1 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:03.127173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a163ffd-d189-404f-a2c2-c22fb7521bc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:03.132877 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a163ffd-d189-404f-a2c2-c22fb7521bc1 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:03.204485 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d3e5a-8d5d-4976-bf3d-1c1f25f53a9f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:03.215978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 987/1954] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:02 2022] POST /identity/v3/users => generated 292 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:51:03.220213 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a163ffd-d189-404f-a2c2-c22fb7521bc1 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:03.245454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a163ffd-d189-404f-a2c2-c22fb7521bc1 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:03.260236 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0981e52-adfc-4830-8d56-0a6b260b95a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:03.260914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0981e52-adfc-4830-8d56-0a6b260b95a7 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:03.261715 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0981e52-adfc-4830-8d56-0a6b260b95a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:03.262003 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0981e52-adfc-4830-8d56-0a6b260b95a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:03.265033 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0981e52-adfc-4830-8d56-0a6b260b95a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:03.267275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0981e52-adfc-4830-8d56-0a6b260b95a7 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:03.295447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 988/1955] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:03.295447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0981e52-adfc-4830-8d56-0a6b260b95a7 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:03.302575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e0981e52-adfc-4830-8d56-0a6b260b95a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:03.335018 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0981e52-adfc-4830-8d56-0a6b260b95a7 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:03.339099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad44422-3919-45dd-82a4-882217c79fc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:03.340129 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad44422-3919-45dd-82a4-882217c79fc2 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:03.340710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad44422-3919-45dd-82a4-882217c79fc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:03.341221 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad44422-3919-45dd-82a4-882217c79fc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:03.349422 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bfba0-4461-46bc-90c7-ad5fdad3f89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:03.350114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bfba0-4461-46bc-90c7-ad5fdad3f89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:03.350401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bfba0-4461-46bc-90c7-ad5fdad3f89c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:03.350682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bfba0-4461-46bc-90c7-ad5fdad3f89c admin admin] PATH_INFO: `/v3/projects/5fb46b96bd3149f8b86e023b6cf34dc3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:03.355486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049bfba0-4461-46bc-90c7-ad5fdad3f89c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fb46b96bd3149f8b86e023b6cf34dc3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:03.358059 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad44422-3919-45dd-82a4-882217c79fc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:03.360404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad44422-3919-45dd-82a4-882217c79fc2 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:03.361116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049bfba0-4461-46bc-90c7-ad5fdad3f89c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:03.376094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 968/1956] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:51:03.378677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad44422-3919-45dd-82a4-882217c79fc2 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:03.395616 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad44422-3919-45dd-82a4-882217c79fc2 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:03.401026 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b27c12-edbd-4ff7-ae81-7e431d689869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:03.401923 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b27c12-edbd-4ff7-ae81-7e431d689869 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:03.402399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b27c12-edbd-4ff7-ae81-7e431d689869 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:03.402944 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b27c12-edbd-4ff7-ae81-7e431d689869 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:03.405557 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b27c12-edbd-4ff7-ae81-7e431d689869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:03.407855 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b27c12-edbd-4ff7-ae81-7e431d689869 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:03.423798 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 969/1957] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:03.428815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b27c12-edbd-4ff7-ae81-7e431d689869 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:03.444190 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b27c12-edbd-4ff7-ae81-7e431d689869 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:03.452737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32340e-df30-41b8-8b6e-cdafca827af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:03.453808 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32340e-df30-41b8-8b6e-cdafca827af9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:03.454322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32340e-df30-41b8-8b6e-cdafca827af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:03.454819 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32340e-df30-41b8-8b6e-cdafca827af9 admin admin] PATH_INFO: `/v3/projects/b647c278c63e44148f8bbe0cf8fad86a/users/56c518fb89dd4686aa1baabeef727661/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:03.484874 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-049bfba0-4461-46bc-90c7-ad5fdad3f89c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fb46b96bd3149f8b86e023b6cf34dc3', 'request_id': 'req-049bfba0-4461-46bc-90c7-ad5fdad3f89c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:03.506843 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32340e-df30-41b8-8b6e-cdafca827af9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b647c278c63e44148f8bbe0cf8fad86a, user_id=56c518fb89dd4686aa1baabeef727661, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:03.517952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32340e-df30-41b8-8b6e-cdafca827af9 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:03.521148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-049bfba0-4461-46bc-90c7-ad5fdad3f89c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fb46b96bd3149f8b86e023b6cf34dc3', 'request_id': 'req-049bfba0-4461-46bc-90c7-ad5fdad3f89c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:03.562826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-049bfba0-4461-46bc-90c7-ad5fdad3f89c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fb46b96bd3149f8b86e023b6cf34dc3', 'request_id': 'req-049bfba0-4461-46bc-90c7-ad5fdad3f89c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:03.572670 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 970/1958] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:03 2022] PUT /identity/v3/projects/b647c278c63e44148f8bbe0cf8fad86a/users/56c518fb89dd4686aa1baabeef727661/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:03.599599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd8a14-1db9-4f99-a404-044ce7905b18 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:03.600002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd8a14-1db9-4f99-a404-044ce7905b18 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:03.600172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd8a14-1db9-4f99-a404-044ce7905b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:03.604033 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-049bfba0-4461-46bc-90c7-ad5fdad3f89c admin admin] The token cache is being invalidate because project 5fb46b96bd3149f8b86e023b6cf34dc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:03.604361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-049bfba0-4461-46bc-90c7-ad5fdad3f89c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:03.608040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-049bfba0-4461-46bc-90c7-ad5fdad3f89c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:03.610058 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 989/1959] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:03 2022] DELETE /identity/v3/projects/5fb46b96bd3149f8b86e023b6cf34dc3 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:03.617005 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bfba0-4461-46bc-90c7-ad5fdad3f89c admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:03.618480 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-049bfba0-4461-46bc-90c7-ad5fdad3f89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:03.689269 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-7cbd8a14-1db9-4f99-a404-044ce7905b18 None None] MFA Rules not processed for user `56c518fb89dd4686aa1baabeef727661`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:03.690604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-7cbd8a14-1db9-4f99-a404-044ce7905b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:03.714184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:03.770401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bfba0-4461-46bc-90c7-ad5fdad3f89c admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:03.791933 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 971/1960] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:03.791933 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd8a14-1db9-4f99-a404-044ce7905b18 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:03.791933 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-7cbd8a14-1db9-4f99-a404-044ce7905b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:03.797251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:03.852680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8b9e4-40c3-4dfa-a71d-0ce230f0aa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:03.853595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8b9e4-40c3-4dfa-a71d-0ce230f0aa2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:03.854028 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8b9e4-40c3-4dfa-a71d-0ce230f0aa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:03.854489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8b9e4-40c3-4dfa-a71d-0ce230f0aa2d admin admin] PATH_INFO: `/v3/users/d7544836f55a49d0a6a966752898be76` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:03.855842 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8b9e4-40c3-4dfa-a71d-0ce230f0aa2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7544836f55a49d0a6a966752898be76)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:03.858690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8b9e4-40c3-4dfa-a71d-0ce230f0aa2d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:03.898111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd8a14-1db9-4f99-a404-044ce7905b18 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:03.907392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa07bd-5552-48bd-9e9b-b076a5525282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:03.908160 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa07bd-5552-48bd-9e9b-b076a5525282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:03.908577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa07bd-5552-48bd-9e9b-b076a5525282 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:03.908891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa07bd-5552-48bd-9e9b-b076a5525282 admin admin] PATH_INFO: `/v3/projects/ed690fe3c501418dbe3703cccc3e17d2` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:03.910171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa07bd-5552-48bd-9e9b-b076a5525282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed690fe3c501418dbe3703cccc3e17d2)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:03.916610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa07bd-5552-48bd-9e9b-b076a5525282 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:03.928225 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-98b8b9e4-40c3-4dfa-a71d-0ce230f0aa2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7544836f55a49d0a6a966752898be76'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:03.947724 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-98b8b9e4-40c3-4dfa-a71d-0ce230f0aa2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7544836f55a49d0a6a966752898be76'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:03.983591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-98b8b9e4-40c3-4dfa-a71d-0ce230f0aa2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7544836f55a49d0a6a966752898be76'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.016381 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-98b8b9e4-40c3-4dfa-a71d-0ce230f0aa2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7544836f55a49d0a6a966752898be76'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.017807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-98b8b9e4-40c3-4dfa-a71d-0ce230f0aa2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7544836f55a49d0a6a966752898be76'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.020483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f3fa07bd-5552-48bd-9e9b-b076a5525282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed690fe3c501418dbe3703cccc3e17d2', 'request_id': 'req-f3fa07bd-5552-48bd-9e9b-b076a5525282'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.026183 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f3fa07bd-5552-48bd-9e9b-b076a5525282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed690fe3c501418dbe3703cccc3e17d2', 'request_id': 'req-f3fa07bd-5552-48bd-9e9b-b076a5525282'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.034286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f3fa07bd-5552-48bd-9e9b-b076a5525282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed690fe3c501418dbe3703cccc3e17d2', 'request_id': 'req-f3fa07bd-5552-48bd-9e9b-b076a5525282'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.039115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 990/1961] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:03 2022] DELETE /identity/v3/users/d7544836f55a49d0a6a966752898be76 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:04.040777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8b9e4-40c3-4dfa-a71d-0ce230f0aa2d admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:04.043042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-98b8b9e4-40c3-4dfa-a71d-0ce230f0aa2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:04.052638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f3fa07bd-5552-48bd-9e9b-b076a5525282 admin admin] The token cache is being invalidate because project ed690fe3c501418dbe3703cccc3e17d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:04.053058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f3fa07bd-5552-48bd-9e9b-b076a5525282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.054928 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-f3fa07bd-5552-48bd-9e9b-b076a5525282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.056638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 972/1962] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:03 2022] DELETE /identity/v3/projects/ed690fe3c501418dbe3703cccc3e17d2 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:04.057506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa07bd-5552-48bd-9e9b-b076a5525282 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:04.060902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f3fa07bd-5552-48bd-9e9b-b076a5525282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:04.124734 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:04.147667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8b9e4-40c3-4dfa-a71d-0ce230f0aa2d admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:04.156460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e372e73-db4a-41df-923e-672af8eeb540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:04.157260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e372e73-db4a-41df-923e-672af8eeb540 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:04.157541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e372e73-db4a-41df-923e-672af8eeb540 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:04.157890 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e372e73-db4a-41df-923e-672af8eeb540 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:04.158625 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e372e73-db4a-41df-923e-672af8eeb540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:04.160471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e372e73-db4a-41df-923e-672af8eeb540 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:04.160789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa07bd-5552-48bd-9e9b-b076a5525282 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:04.166485 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2260b-525f-4329-9fd6-ea038630acf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:04.167426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2260b-525f-4329-9fd6-ea038630acf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:04.167814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2260b-525f-4329-9fd6-ea038630acf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:04.168195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2260b-525f-4329-9fd6-ea038630acf1 admin admin] PATH_INFO: `/v3/projects/401d2ab88cf346cc86ecf524effbdb2d` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:04.170377 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2260b-525f-4329-9fd6-ea038630acf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=401d2ab88cf346cc86ecf524effbdb2d)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:04.173688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2260b-525f-4329-9fd6-ea038630acf1 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:04.203027 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 991/1963] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:04 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:04.204333 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e372e73-db4a-41df-923e-672af8eeb540 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:04.206678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0e372e73-db4a-41df-923e-672af8eeb540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:04.260617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:04.271568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c6b2260b-525f-4329-9fd6-ea038630acf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '401d2ab88cf346cc86ecf524effbdb2d', 'request_id': 'req-c6b2260b-525f-4329-9fd6-ea038630acf1'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.296914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c6b2260b-525f-4329-9fd6-ea038630acf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '401d2ab88cf346cc86ecf524effbdb2d', 'request_id': 'req-c6b2260b-525f-4329-9fd6-ea038630acf1'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.309789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c6b2260b-525f-4329-9fd6-ea038630acf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '401d2ab88cf346cc86ecf524effbdb2d', 'request_id': 'req-c6b2260b-525f-4329-9fd6-ea038630acf1'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.330465 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e372e73-db4a-41df-923e-672af8eeb540 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:04.338377 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c6b2260b-525f-4329-9fd6-ea038630acf1 admin admin] The token cache is being invalidate because project 401d2ab88cf346cc86ecf524effbdb2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:04.338676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c6b2260b-525f-4329-9fd6-ea038630acf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.340705 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c6b2260b-525f-4329-9fd6-ea038630acf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.344560 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 973/1964] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:04 2022] DELETE /identity/v3/projects/401d2ab88cf346cc86ecf524effbdb2d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:04.345119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2260b-525f-4329-9fd6-ea038630acf1 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:04.353018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c6b2260b-525f-4329-9fd6-ea038630acf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:04.416218 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f5fd6-9d6b-4e30-98ea-50e268940109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:04.417241 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f5fd6-9d6b-4e30-98ea-50e268940109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:04.417640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f5fd6-9d6b-4e30-98ea-50e268940109 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:04.418268 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f5fd6-9d6b-4e30-98ea-50e268940109 admin admin] PATH_INFO: `/v3/users/bd014fe75d384061a2e56e875401f4d9` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:04.421839 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f5fd6-9d6b-4e30-98ea-50e268940109 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd014fe75d384061a2e56e875401f4d9)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:04.424507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f5fd6-9d6b-4e30-98ea-50e268940109 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:04.454736 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2260b-525f-4329-9fd6-ea038630acf1 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:04.460453 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc266aa1-4c40-4e44-b386-9f2594c0c7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:04.461514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc266aa1-4c40-4e44-b386-9f2594c0c7f2 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:04.462232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc266aa1-4c40-4e44-b386-9f2594c0c7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:04.462539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc266aa1-4c40-4e44-b386-9f2594c0c7f2 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:04.463740 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc266aa1-4c40-4e44-b386-9f2594c0c7f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:04.466258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc266aa1-4c40-4e44-b386-9f2594c0c7f2 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:04.493323 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 974/1965] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:04 2022] POST /identity/v3/users => generated 298 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:04.494881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc266aa1-4c40-4e44-b386-9f2594c0c7f2 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:04.498368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-bc266aa1-4c40-4e44-b386-9f2594c0c7f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:04.500581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5a3f5fd6-9d6b-4e30-98ea-50e268940109 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd014fe75d384061a2e56e875401f4d9'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.511416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5a3f5fd6-9d6b-4e30-98ea-50e268940109 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd014fe75d384061a2e56e875401f4d9'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.513302 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc266aa1-4c40-4e44-b386-9f2594c0c7f2 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:04.519169 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55aae5-920d-4292-9aaf-2394ea454c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:04.519824 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55aae5-920d-4292-9aaf-2394ea454c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:04.520108 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55aae5-920d-4292-9aaf-2394ea454c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:04.520419 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55aae5-920d-4292-9aaf-2394ea454c38 admin admin] PATH_INFO: `/v3/users/63ab4e2c8a844ed199a512862693dd5a` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:04.521601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55aae5-920d-4292-9aaf-2394ea454c38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ab4e2c8a844ed199a512862693dd5a)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:04.523504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55aae5-920d-4292-9aaf-2394ea454c38 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:04.528735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5a3f5fd6-9d6b-4e30-98ea-50e268940109 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd014fe75d384061a2e56e875401f4d9'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.541219 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5a3f5fd6-9d6b-4e30-98ea-50e268940109 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd014fe75d384061a2e56e875401f4d9'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.542075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5a3f5fd6-9d6b-4e30-98ea-50e268940109 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd014fe75d384061a2e56e875401f4d9'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.552593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 992/1966] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:04 2022] DELETE /identity/v3/users/bd014fe75d384061a2e56e875401f4d9 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:04.552593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f5fd6-9d6b-4e30-98ea-50e268940109 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:04.552593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5a3f5fd6-9d6b-4e30-98ea-50e268940109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:04.619766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-dd55aae5-920d-4292-9aaf-2394ea454c38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ab4e2c8a844ed199a512862693dd5a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.636803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-dd55aae5-920d-4292-9aaf-2394ea454c38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ab4e2c8a844ed199a512862693dd5a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.651252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=79,keystone:DELETE=16,keystone:INSERT=5 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:04.665613 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-dd55aae5-920d-4292-9aaf-2394ea454c38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ab4e2c8a844ed199a512862693dd5a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.679290 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:04.681776 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-dd55aae5-920d-4292-9aaf-2394ea454c38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ab4e2c8a844ed199a512862693dd5a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.684514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-dd55aae5-920d-4292-9aaf-2394ea454c38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ab4e2c8a844ed199a512862693dd5a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.714150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 975/1967] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:04 2022] DELETE /identity/v3/users/63ab4e2c8a844ed199a512862693dd5a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:04.720076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55aae5-920d-4292-9aaf-2394ea454c38 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:04.721332 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-dd55aae5-920d-4292-9aaf-2394ea454c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:04.727721 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f5fd6-9d6b-4e30-98ea-50e268940109 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:04.821715 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48799f93-6fa5-41ca-9020-18f4546262fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:04.822358 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48799f93-6fa5-41ca-9020-18f4546262fa admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:04.822646 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48799f93-6fa5-41ca-9020-18f4546262fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:04.822927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48799f93-6fa5-41ca-9020-18f4546262fa admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:04.823826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48799f93-6fa5-41ca-9020-18f4546262fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:04.825663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48799f93-6fa5-41ca-9020-18f4546262fa admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:04.827459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55aae5-920d-4292-9aaf-2394ea454c38 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:04.838209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677b9c4-e089-40d8-918a-7aaa20163237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:04.839185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677b9c4-e089-40d8-918a-7aaa20163237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:04.839621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677b9c4-e089-40d8-918a-7aaa20163237 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:04.840040 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677b9c4-e089-40d8-918a-7aaa20163237 admin admin] PATH_INFO: `/v3/projects/692c4ef4817a40ada6c1c312af107ed2` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:04.842579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677b9c4-e089-40d8-918a-7aaa20163237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=692c4ef4817a40ada6c1c312af107ed2)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:04.845228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677b9c4-e089-40d8-918a-7aaa20163237 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:04.851886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 993/1968] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:04.856571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48799f93-6fa5-41ca-9020-18f4546262fa admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:04.859870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-48799f93-6fa5-41ca-9020-18f4546262fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:04.882053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48799f93-6fa5-41ca-9020-18f4546262fa admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:04.889232 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c0ace-bb3c-410f-88ba-12331126f69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:04.890105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c0ace-bb3c-410f-88ba-12331126f69e admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:04.890390 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c0ace-bb3c-410f-88ba-12331126f69e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:04.890677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c0ace-bb3c-410f-88ba-12331126f69e admin admin] PATH_INFO: `/v3/projects/1685a87fa202447996decc9c3371d22d/users/eb21d5dd2412459ea6ccaa9d707e6b51/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:04.913700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c0ace-bb3c-410f-88ba-12331126f69e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1685a87fa202447996decc9c3371d22d, user_id=eb21d5dd2412459ea6ccaa9d707e6b51, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:04.916528 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c0ace-bb3c-410f-88ba-12331126f69e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:04.925396 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3677b9c4-e089-40d8-918a-7aaa20163237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '692c4ef4817a40ada6c1c312af107ed2', 'request_id': 'req-3677b9c4-e089-40d8-918a-7aaa20163237'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.940950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3677b9c4-e089-40d8-918a-7aaa20163237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '692c4ef4817a40ada6c1c312af107ed2', 'request_id': 'req-3677b9c4-e089-40d8-918a-7aaa20163237'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.942163 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 994/1969] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:04 2022] PUT /identity/v3/projects/1685a87fa202447996decc9c3371d22d/users/eb21d5dd2412459ea6ccaa9d707e6b51/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:04.945772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c0ace-bb3c-410f-88ba-12331126f69e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:04.954440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3677b9c4-e089-40d8-918a-7aaa20163237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '692c4ef4817a40ada6c1c312af107ed2', 'request_id': 'req-3677b9c4-e089-40d8-918a-7aaa20163237'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.977579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3677b9c4-e089-40d8-918a-7aaa20163237 admin admin] The token cache is being invalidate because project 692c4ef4817a40ada6c1c312af107ed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:04.978147 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3677b9c4-e089-40d8-918a-7aaa20163237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.979026 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3677b9c4-e089-40d8-918a-7aaa20163237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:04.981337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 976/1970] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:04 2022] DELETE /identity/v3/projects/692c4ef4817a40ada6c1c312af107ed2 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:04.987053 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64520e-26d1-425c-84ba-ce645f44b5fa None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:04.987790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64520e-26d1-425c-84ba-ce645f44b5fa None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:04.988372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64520e-26d1-425c-84ba-ce645f44b5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.032903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-4e64520e-26d1-425c-84ba-ce645f44b5fa None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:05.034291 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4e64520e-26d1-425c-84ba-ce645f44b5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:05.051996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 977/1971] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:05.052544 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c0ace-bb3c-410f-88ba-12331126f69e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:05.062164 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848c755-6218-4ec0-bcf0-edd31f964b14 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.062442 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848c755-6218-4ec0-bcf0-edd31f964b14 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.062776 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848c755-6218-4ec0-bcf0-edd31f964b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.064525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae137a4-9461-497c-8e58-72da1dda9e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:05.065271 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae137a4-9461-497c-8e58-72da1dda9e1a admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.065682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae137a4-9461-497c-8e58-72da1dda9e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.065951 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae137a4-9461-497c-8e58-72da1dda9e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.067298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae137a4-9461-497c-8e58-72da1dda9e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:05.069055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae137a4-9461-497c-8e58-72da1dda9e1a admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:05.083698 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 995/1972] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:51:05.087652 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae137a4-9461-497c-8e58-72da1dda9e1a admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:05.090748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8ae137a4-9461-497c-8e58-72da1dda9e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:05.093476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-b848c755-6218-4ec0-bcf0-edd31f964b14 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:05.094575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b848c755-6218-4ec0-bcf0-edd31f964b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:05.106357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 978/1973] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:05.108661 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae137a4-9461-497c-8e58-72da1dda9e1a admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:05.115390 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809dc666-833a-4991-905f-80139d924c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:05.116093 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dc666-833a-4991-905f-80139d924c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.116385 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dc666-833a-4991-905f-80139d924c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.116683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dc666-833a-4991-905f-80139d924c1c admin admin] PATH_INFO: `/v3/projects/1685a87fa202447996decc9c3371d22d/users/eb21d5dd2412459ea6ccaa9d707e6b51/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.120038 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848c755-6218-4ec0-bcf0-edd31f964b14 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:05.121539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809dc666-833a-4991-905f-80139d924c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1685a87fa202447996decc9c3371d22d, user_id=eb21d5dd2412459ea6ccaa9d707e6b51, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:05.128894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809dc666-833a-4991-905f-80139d924c1c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:05.142472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 996/1974] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:05 2022] PUT /identity/v3/projects/1685a87fa202447996decc9c3371d22d/users/eb21d5dd2412459ea6ccaa9d707e6b51/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:05.152755 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809dc666-833a-4991-905f-80139d924c1c admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:05.157200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848c755-6218-4ec0-bcf0-edd31f964b14 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:05.157804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-809dc666-833a-4991-905f-80139d924c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:05.229710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:05.239370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b00984-4f15-4478-966e-5e78f55a9b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:05.240480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b00984-4f15-4478-966e-5e78f55a9b26 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.240948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b00984-4f15-4478-966e-5e78f55a9b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.242194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b00984-4f15-4478-966e-5e78f55a9b26 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.243169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:05.243591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b00984-4f15-4478-966e-5e78f55a9b26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:51:05.244101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b00984-4f15-4478-966e-5e78f55a9b26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:05.247138 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b00984-4f15-4478-966e-5e78f55a9b26 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:05.256054 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809dc666-833a-4991-905f-80139d924c1c admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:05.266227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:05.267201 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 979/1975] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:51:05.267386 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.267784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.269342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8 admin admin] PATH_INFO: `/v3/projects/aef95145859544428e263817cfdd9c56` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.272189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aef95145859544428e263817cfdd9c56)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:05.272467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0ec32-de2e-46e8-a25c-c7c5dce8686a None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.272759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0ec32-de2e-46e8-a25c-c7c5dce8686a None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.273036 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0ec32-de2e-46e8-a25c-c7c5dce8686a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.274756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:05.310420 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ead0ec32-de2e-46e8-a25c-c7c5dce8686a None None] MFA Rules not processed for user `eb21d5dd2412459ea6ccaa9d707e6b51`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:05.312613 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ead0ec32-de2e-46e8-a25c-c7c5dce8686a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:05.377006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aef95145859544428e263817cfdd9c56', 'request_id': 'req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:05.389413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aef95145859544428e263817cfdd9c56', 'request_id': 'req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:05.402644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aef95145859544428e263817cfdd9c56', 'request_id': 'req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:05.415313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 980/1976] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:05.416160 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0ec32-de2e-46e8-a25c-c7c5dce8686a None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:05.416882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8 admin admin] The token cache is being invalidate because project aef95145859544428e263817cfdd9c56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:05.417170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:05.418052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:05.434837 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 997/1977] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:05 2022] DELETE /identity/v3/projects/aef95145859544428e263817cfdd9c56 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:05.435029 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0ec32-de2e-46e8-a25c-c7c5dce8686a None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:05.435112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:05.435112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:05.525738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7c600-fe17-4d75-bdf6-4de0529e6eb8 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:05.526788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502837e-44c9-47cd-a9ea-bc5e65660b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:05.527564 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502837e-44c9-47cd-a9ea-bc5e65660b68 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.527924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502837e-44c9-47cd-a9ea-bc5e65660b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.528269 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502837e-44c9-47cd-a9ea-bc5e65660b68 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.529089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502837e-44c9-47cd-a9ea-bc5e65660b68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:05.530878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2834d4a-fcfe-42b3-9996-0eef0eb5783e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:05.531527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502837e-44c9-47cd-a9ea-bc5e65660b68 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:05.532209 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2834d4a-fcfe-42b3-9996-0eef0eb5783e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.532543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2834d4a-fcfe-42b3-9996-0eef0eb5783e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.532831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2834d4a-fcfe-42b3-9996-0eef0eb5783e admin admin] PATH_INFO: `/v3/projects/8c2191c9560d4a8b8275ab95837731c2` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.534060 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2834d4a-fcfe-42b3-9996-0eef0eb5783e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c2191c9560d4a8b8275ab95837731c2)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:05.536436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2834d4a-fcfe-42b3-9996-0eef0eb5783e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:05.577137 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 981/1978] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:05 2022] POST /identity/v3/projects => generated 409 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:05.577137 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502837e-44c9-47cd-a9ea-bc5e65660b68 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:05.579485 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f502837e-44c9-47cd-a9ea-bc5e65660b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:05.600365 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e2834d4a-fcfe-42b3-9996-0eef0eb5783e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c2191c9560d4a8b8275ab95837731c2', 'request_id': 'req-e2834d4a-fcfe-42b3-9996-0eef0eb5783e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:05.607545 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e2834d4a-fcfe-42b3-9996-0eef0eb5783e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c2191c9560d4a8b8275ab95837731c2', 'request_id': 'req-e2834d4a-fcfe-42b3-9996-0eef0eb5783e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:05.617119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e2834d4a-fcfe-42b3-9996-0eef0eb5783e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c2191c9560d4a8b8275ab95837731c2', 'request_id': 'req-e2834d4a-fcfe-42b3-9996-0eef0eb5783e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:05.634733 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e2834d4a-fcfe-42b3-9996-0eef0eb5783e admin admin] The token cache is being invalidate because project 8c2191c9560d4a8b8275ab95837731c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:05.635394 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e2834d4a-fcfe-42b3-9996-0eef0eb5783e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:05.636773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e2834d4a-fcfe-42b3-9996-0eef0eb5783e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:05.640722 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 998/1979] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:05 2022] DELETE /identity/v3/projects/8c2191c9560d4a8b8275ab95837731c2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:05.643971 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557e963-338c-4625-87db-bd8afcd45f7a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.644734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557e963-338c-4625-87db-bd8afcd45f7a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.645772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557e963-338c-4625-87db-bd8afcd45f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.655034 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502837e-44c9-47cd-a9ea-bc5e65660b68 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:05.669529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:05.671328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-a557e963-338c-4625-87db-bd8afcd45f7a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:05.672670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a557e963-338c-4625-87db-bd8afcd45f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:05.683317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:05.719909 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6926b-23dc-407e-b987-a601a1b32670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:05.720650 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6926b-23dc-407e-b987-a601a1b32670 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.720943 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6926b-23dc-407e-b987-a601a1b32670 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.721251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6926b-23dc-407e-b987-a601a1b32670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.723890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-09c6926b-23dc-407e-b987-a601a1b32670 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:05.759150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 999/1980] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:05.761469 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557e963-338c-4625-87db-bd8afcd45f7a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:05.764427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a557e963-338c-4625-87db-bd8afcd45f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:05.779702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557e963-338c-4625-87db-bd8afcd45f7a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:05.784976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42766a91-55b9-4e5c-b180-e9581c62ce5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:05.785676 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42766a91-55b9-4e5c-b180-e9581c62ce5d admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.785968 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42766a91-55b9-4e5c-b180-e9581c62ce5d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.786266 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42766a91-55b9-4e5c-b180-e9581c62ce5d admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.786937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42766a91-55b9-4e5c-b180-e9581c62ce5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:05.789050 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42766a91-55b9-4e5c-b180-e9581c62ce5d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:05.803376 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6926b-23dc-407e-b987-a601a1b32670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:05.804903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6926b-23dc-407e-b987-a601a1b32670 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:05.814480 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1000/1981] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:05 2022] POST /identity/v3/users => generated 305 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:05.816141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8def26-78f9-4825-bc99-ac8cb650f627 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.816585 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8def26-78f9-4825-bc99-ac8cb650f627 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.816937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8def26-78f9-4825-bc99-ac8cb650f627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.818479 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 982/1982] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:05.819698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde51744-1d11-4a8c-b7c4-7907cb76734e None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.819978 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde51744-1d11-4a8c-b7c4-7907cb76734e None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.820326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde51744-1d11-4a8c-b7c4-7907cb76734e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.842530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-2a8def26-78f9-4825-bc99-ac8cb650f627 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:05.844344 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2a8def26-78f9-4825-bc99-ac8cb650f627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:05.845212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-cde51744-1d11-4a8c-b7c4-7907cb76734e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:05.846435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-cde51744-1d11-4a8c-b7c4-7907cb76734e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:05.855115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 983/1983] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:05.859661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde51744-1d11-4a8c-b7c4-7907cb76734e None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:05.860053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1001/1984] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:05.860932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8def26-78f9-4825-bc99-ac8cb650f627 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:05.869999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde51744-1d11-4a8c-b7c4-7907cb76734e None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:05.871172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8def26-78f9-4825-bc99-ac8cb650f627 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:05.876230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fff86-99f4-49cf-9a22-46da3a77d126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:05.876682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190060d-820c-4447-a956-146212eb164c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:05.876883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fff86-99f4-49cf-9a22-46da3a77d126 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.877129 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fff86-99f4-49cf-9a22-46da3a77d126 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.877300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190060d-820c-4447-a956-146212eb164c admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.877398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fff86-99f4-49cf-9a22-46da3a77d126 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.877668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190060d-820c-4447-a956-146212eb164c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.878077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190060d-820c-4447-a956-146212eb164c admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.878196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fff86-99f4-49cf-9a22-46da3a77d126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:05.878792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190060d-820c-4447-a956-146212eb164c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:51:05.879358 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190060d-820c-4447-a956-146212eb164c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:05.879957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fff86-99f4-49cf-9a22-46da3a77d126 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:05.881282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190060d-820c-4447-a956-146212eb164c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:05.892516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 984/1985] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:05.896342 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceead67c-4287-4990-8c44-0cc939958730 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.896710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceead67c-4287-4990-8c44-0cc939958730 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.898000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceead67c-4287-4990-8c44-0cc939958730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.903614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1002/1986] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:51:05.905998 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190060d-820c-4447-a956-146212eb164c admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:05.915917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190060d-820c-4447-a956-146212eb164c admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:05.921769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a2f02-bf70-4d19-9f67-5a6099b9beaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:05.922444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a2f02-bf70-4d19-9f67-5a6099b9beaa admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.922768 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a2f02-bf70-4d19-9f67-5a6099b9beaa admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.923071 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a2f02-bf70-4d19-9f67-5a6099b9beaa admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.923757 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a2f02-bf70-4d19-9f67-5a6099b9beaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:05.925404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a2f02-bf70-4d19-9f67-5a6099b9beaa admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:05.927833 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ceead67c-4287-4990-8c44-0cc939958730 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:05.929252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ceead67c-4287-4990-8c44-0cc939958730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:05.937585 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1003/1987] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:05.938839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 985/1988] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:05.941569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a2f02-bf70-4d19-9f67-5a6099b9beaa admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:05.947981 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceead67c-4287-4990-8c44-0cc939958730 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:05.952813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a2f02-bf70-4d19-9f67-5a6099b9beaa admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:05.955675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceead67c-4287-4990-8c44-0cc939958730 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:05.958065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f422e31-be9a-424f-b8dc-6d6884628e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:05.958690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f422e31-be9a-424f-b8dc-6d6884628e9b admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.958976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f422e31-be9a-424f-b8dc-6d6884628e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.959255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f422e31-be9a-424f-b8dc-6d6884628e9b admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.959927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f422e31-be9a-424f-b8dc-6d6884628e9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:05.960682 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cd38a-3f47-45d2-81d9-7729dd3f28a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:05.961384 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cd38a-3f47-45d2-81d9-7729dd3f28a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:05.961663 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cd38a-3f47-45d2-81d9-7729dd3f28a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:05.961868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f422e31-be9a-424f-b8dc-6d6884628e9b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:05.961959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cd38a-3f47-45d2-81d9-7729dd3f28a5 admin admin] PATH_INFO: `/v3/projects/238d8bcf271a4134b5cf3c83b9e9c87b/users/2a1bd85a6f8540129a4e3ebd096c64b2/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:05.977729 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cd38a-3f47-45d2-81d9-7729dd3f28a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=238d8bcf271a4134b5cf3c83b9e9c87b, user_id=2a1bd85a6f8540129a4e3ebd096c64b2, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:05.980571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cd38a-3f47-45d2-81d9-7729dd3f28a5 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:05.990342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1004/1989] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:05 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:05.991261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f422e31-be9a-424f-b8dc-6d6884628e9b admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:05.995453 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 986/1990] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:05 2022] PUT /identity/v3/projects/238d8bcf271a4134b5cf3c83b9e9c87b/users/2a1bd85a6f8540129a4e3ebd096c64b2/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:05.996261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cd38a-3f47-45d2-81d9-7729dd3f28a5 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:05.998560 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-285cd38a-3f47-45d2-81d9-7729dd3f28a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:06.062768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cd38a-3f47-45d2-81d9-7729dd3f28a5 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.065465 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f422e31-be9a-424f-b8dc-6d6884628e9b admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.072037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cf966-f9ca-4088-a15e-572d4940e5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:06.072831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cf966-f9ca-4088-a15e-572d4940e5bc admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.073800 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cf966-f9ca-4088-a15e-572d4940e5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.074093 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cf966-f9ca-4088-a15e-572d4940e5bc admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.074860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631cf966-f9ca-4088-a15e-572d4940e5bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:51:06.075170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631cf966-f9ca-4088-a15e-572d4940e5bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:06.077880 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631cf966-f9ca-4088-a15e-572d4940e5bc admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:06.093189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1005/1991] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:51:06.098101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cf966-f9ca-4088-a15e-572d4940e5bc admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:06.101858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:06.112902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cf966-f9ca-4088-a15e-572d4940e5bc admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.121216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b658e-ea82-4ffb-aacd-b41ee866f3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:06.122100 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b658e-ea82-4ffb-aacd-b41ee866f3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.122379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b658e-ea82-4ffb-aacd-b41ee866f3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.123487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b658e-ea82-4ffb-aacd-b41ee866f3d0 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.124394 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b658e-ea82-4ffb-aacd-b41ee866f3d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:06.126619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b658e-ea82-4ffb-aacd-b41ee866f3d0 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:06.129481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833fa34-48f1-42ef-9ec7-fe6488418e9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:06.130132 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833fa34-48f1-42ef-9ec7-fe6488418e9b service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.130418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833fa34-48f1-42ef-9ec7-fe6488418e9b service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.130695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833fa34-48f1-42ef-9ec7-fe6488418e9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.132812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c833fa34-48f1-42ef-9ec7-fe6488418e9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:06.153068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1006/1992] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:06 2022] POST /identity/v3/users => generated 291 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:06.157382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbcb82-83f0-4119-b7e9-d893597f2627 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.157911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbcb82-83f0-4119-b7e9-d893597f2627 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.158198 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbcb82-83f0-4119-b7e9-d893597f2627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.191813 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-66fbcb82-83f0-4119-b7e9-d893597f2627 None None] MFA Rules not processed for user `2a1bd85a6f8540129a4e3ebd096c64b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:06.192993 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-66fbcb82-83f0-4119-b7e9-d893597f2627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:06.203505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:06.205952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833fa34-48f1-42ef-9ec7-fe6488418e9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:06.207960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833fa34-48f1-42ef-9ec7-fe6488418e9b service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:06.220557 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 987/1993] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:06.228986 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833fa34-48f1-42ef-9ec7-fe6488418e9b service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:06.246333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833fa34-48f1-42ef-9ec7-fe6488418e9b service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.254834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73561b-3400-4e00-b550-234aeacfecb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:06.255580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73561b-3400-4e00-b550-234aeacfecb2 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.255964 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73561b-3400-4e00-b550-234aeacfecb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.256374 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73561b-3400-4e00-b550-234aeacfecb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.257309 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73561b-3400-4e00-b550-234aeacfecb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:06.259454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73561b-3400-4e00-b550-234aeacfecb2 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:06.292268 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1007/1994] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:06 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:06.294317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbcb82-83f0-4119-b7e9-d893597f2627 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:06.299157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 988/1995] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:06 2022] POST /identity/v3/projects => generated 361 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:06.300479 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73561b-3400-4e00-b550-234aeacfecb2 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:06.326892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbcb82-83f0-4119-b7e9-d893597f2627 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.405108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafb61d-121a-41e6-bc3c-4036b83d01a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:06.405745 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafb61d-121a-41e6-bc3c-4036b83d01a4 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.406022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafb61d-121a-41e6-bc3c-4036b83d01a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.406398 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafb61d-121a-41e6-bc3c-4036b83d01a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.406998 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafb61d-121a-41e6-bc3c-4036b83d01a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:06.408694 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafb61d-121a-41e6-bc3c-4036b83d01a4 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:06.412465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73561b-3400-4e00-b550-234aeacfecb2 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.418065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da4b37-fe23-4001-aae8-14e97a25f95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:06.418775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da4b37-fe23-4001-aae8-14e97a25f95e admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.419110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1008/1996] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:06.419228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da4b37-fe23-4001-aae8-14e97a25f95e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.419659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da4b37-fe23-4001-aae8-14e97a25f95e admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.419986 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafb61d-121a-41e6-bc3c-4036b83d01a4 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:06.420685 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da4b37-fe23-4001-aae8-14e97a25f95e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:06.423051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da4b37-fe23-4001-aae8-14e97a25f95e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:06.431356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafb61d-121a-41e6-bc3c-4036b83d01a4 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.437784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0788ea1-0d14-4902-95d1-6f5584e7c1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:06.438515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0788ea1-0d14-4902-95d1-6f5584e7c1d9 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.438862 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0788ea1-0d14-4902-95d1-6f5584e7c1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.439364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0788ea1-0d14-4902-95d1-6f5584e7c1d9 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.440616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0788ea1-0d14-4902-95d1-6f5584e7c1d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:06.442597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0788ea1-0d14-4902-95d1-6f5584e7c1d9 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:06.457712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 989/1997] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:06 2022] POST /identity/v3/projects => generated 407 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:06.458575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da4b37-fe23-4001-aae8-14e97a25f95e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:06.476059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1009/1998] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:06 2022] POST /identity/v3/users => generated 281 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:06.476059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0788ea1-0d14-4902-95d1-6f5584e7c1d9 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:06.485576 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0788ea1-0d14-4902-95d1-6f5584e7c1d9 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.550510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:06.562356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec243a9a-bc9a-4119-8f89-3fd60457e008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:06.562979 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec243a9a-bc9a-4119-8f89-3fd60457e008 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.563260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec243a9a-bc9a-4119-8f89-3fd60457e008 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.563630 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec243a9a-bc9a-4119-8f89-3fd60457e008 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.564221 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec243a9a-bc9a-4119-8f89-3fd60457e008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:06.565939 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da4b37-fe23-4001-aae8-14e97a25f95e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.566147 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec243a9a-bc9a-4119-8f89-3fd60457e008 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:06.570846 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512e34b-8a1c-46d3-947f-f8069306918c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:06.571473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512e34b-8a1c-46d3-947f-f8069306918c admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.571756 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512e34b-8a1c-46d3-947f-f8069306918c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.572033 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512e34b-8a1c-46d3-947f-f8069306918c admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.572792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8512e34b-8a1c-46d3-947f-f8069306918c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:06.574356 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8512e34b-8a1c-46d3-947f-f8069306918c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:06.589548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 990/1999] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:06.590497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512e34b-8a1c-46d3-947f-f8069306918c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:06.600056 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1010/2000] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:06 2022] POST /identity/v3/users => generated 310 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:06.600056 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec243a9a-bc9a-4119-8f89-3fd60457e008 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:06.600056 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ec243a9a-bc9a-4119-8f89-3fd60457e008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:06.605183 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512e34b-8a1c-46d3-947f-f8069306918c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.610549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64708c3-d3d4-43fc-bde2-43166ba9902a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:06.611397 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64708c3-d3d4-43fc-bde2-43166ba9902a admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.611826 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64708c3-d3d4-43fc-bde2-43166ba9902a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.612524 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64708c3-d3d4-43fc-bde2-43166ba9902a admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.612919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64708c3-d3d4-43fc-bde2-43166ba9902a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:06.614623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64708c3-d3d4-43fc-bde2-43166ba9902a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:06.615562 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec243a9a-bc9a-4119-8f89-3fd60457e008 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.620659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3288d7-6173-4d97-b665-f1a379bef9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:06.621343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3288d7-6173-4d97-b665-f1a379bef9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.621627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3288d7-6173-4d97-b665-f1a379bef9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.621903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3288d7-6173-4d97-b665-f1a379bef9fe admin admin] PATH_INFO: `/v3/users/56c518fb89dd4686aa1baabeef727661` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.623339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3288d7-6173-4d97-b665-f1a379bef9fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=56c518fb89dd4686aa1baabeef727661)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:06.625330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3288d7-6173-4d97-b665-f1a379bef9fe admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:06.627084 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 991/2001] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:06.628185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64708c3-d3d4-43fc-bde2-43166ba9902a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:06.638984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64708c3-d3d4-43fc-bde2-43166ba9902a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.644817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:06.644979 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d18512-dec8-495d-a1e7-fc1c680d6ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:06.645913 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d18512-dec8-495d-a1e7-fc1c680d6ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.645913 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d18512-dec8-495d-a1e7-fc1c680d6ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.646296 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d18512-dec8-495d-a1e7-fc1c680d6ecc admin admin] PATH_INFO: `/v3/projects/471e02957ac348929a4da4710b279883/users/b1760aa3b3234380b5b092b4966f3d6a/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.666765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d18512-dec8-495d-a1e7-fc1c680d6ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=471e02957ac348929a4da4710b279883, user_id=b1760aa3b3234380b5b092b4966f3d6a, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:06.669882 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d18512-dec8-495d-a1e7-fc1c680d6ecc admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:06.684743 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 992/2002] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:06 2022] PUT /identity/v3/projects/471e02957ac348929a4da4710b279883/users/b1760aa3b3234380b5b092b4966f3d6a/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:06.693757 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8a3288d7-6173-4d97-b665-f1a379bef9fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56c518fb89dd4686aa1baabeef727661'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:06.693892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d18512-dec8-495d-a1e7-fc1c680d6ecc admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:06.694372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8a3288d7-6173-4d97-b665-f1a379bef9fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56c518fb89dd4686aa1baabeef727661'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:06.710734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8a3288d7-6173-4d97-b665-f1a379bef9fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56c518fb89dd4686aa1baabeef727661'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:06.718942 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8a3288d7-6173-4d97-b665-f1a379bef9fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56c518fb89dd4686aa1baabeef727661'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:06.719697 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8a3288d7-6173-4d97-b665-f1a379bef9fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56c518fb89dd4686aa1baabeef727661'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:06.726291 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1011/2003] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:06 2022] DELETE /identity/v3/users/56c518fb89dd4686aa1baabeef727661 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:06.727106 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3288d7-6173-4d97-b665-f1a379bef9fe admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:06.728470 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8a3288d7-6173-4d97-b665-f1a379bef9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:06.769768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d18512-dec8-495d-a1e7-fc1c680d6ecc admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.802541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3288d7-6173-4d97-b665-f1a379bef9fe admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.807473 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf9603-9952-4422-8f00-312f9bbdfcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:06.808113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9603-9952-4422-8f00-312f9bbdfcf8 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.808453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9603-9952-4422-8f00-312f9bbdfcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.808729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9603-9952-4422-8f00-312f9bbdfcf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.809439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf9603-9952-4422-8f00-312f9bbdfcf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:06.812111 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf9603-9952-4422-8f00-312f9bbdfcf8 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:06.823112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1012/2004] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:06.824588 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56314b95-efe9-4153-9b2b-4aa0968c2386 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.824946 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56314b95-efe9-4153-9b2b-4aa0968c2386 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.825291 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56314b95-efe9-4153-9b2b-4aa0968c2386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.837190 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f8f7d-7b0c-46bf-9a46-a2d5e938e3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:06.837807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f8f7d-7b0c-46bf-9a46-a2d5e938e3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.838088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f8f7d-7b0c-46bf-9a46-a2d5e938e3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.838360 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f8f7d-7b0c-46bf-9a46-a2d5e938e3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.839064 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f8f7d-7b0c-46bf-9a46-a2d5e938e3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:06.840661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f8f7d-7b0c-46bf-9a46-a2d5e938e3f6 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:06.851308 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 993/2005] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:06.852615 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53b889-1543-4001-92d1-e272ad95db91 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.852907 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53b889-1543-4001-92d1-e272ad95db91 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.853179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53b889-1543-4001-92d1-e272ad95db91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.855905 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-56314b95-efe9-4153-9b2b-4aa0968c2386 None None] MFA Rules not processed for user `b1760aa3b3234380b5b092b4966f3d6a`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:06.857665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-56314b95-efe9-4153-9b2b-4aa0968c2386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:06.873728 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-4b53b889-1543-4001-92d1-e272ad95db91 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:06.875138 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4b53b889-1543-4001-92d1-e272ad95db91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:06.883070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 994/2006] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:06.883815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53b889-1543-4001-92d1-e272ad95db91 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:06.893224 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53b889-1543-4001-92d1-e272ad95db91 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.898570 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bab46-5e6f-4c50-9237-c9e5368bb0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:06.899269 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bab46-5e6f-4c50-9237-c9e5368bb0c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:06.899548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bab46-5e6f-4c50-9237-c9e5368bb0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:06.899823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bab46-5e6f-4c50-9237-c9e5368bb0c3 admin admin] PATH_INFO: `/v3/projects/5b9b816a4b694b83a87dbaf44caee57d/users/844bec9cbdd54718981c137d31234a57/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:06.923264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204bab46-5e6f-4c50-9237-c9e5368bb0c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b9b816a4b694b83a87dbaf44caee57d, user_id=844bec9cbdd54718981c137d31234a57, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:06.927701 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204bab46-5e6f-4c50-9237-c9e5368bb0c3 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:06.934724 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1013/2007] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:06.934724 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56314b95-efe9-4153-9b2b-4aa0968c2386 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:06.935966 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-56314b95-efe9-4153-9b2b-4aa0968c2386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:06.952262 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56314b95-efe9-4153-9b2b-4aa0968c2386 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:06.953157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 995/2008] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:06 2022] PUT /identity/v3/projects/5b9b816a4b694b83a87dbaf44caee57d/users/844bec9cbdd54718981c137d31234a57/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:06.965267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bab46-5e6f-4c50-9237-c9e5368bb0c3 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:06.967248 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:06.967860 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-204bab46-5e6f-4c50-9237-c9e5368bb0c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:07.002169 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:07.026312 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e5abb-c922-429d-8c82-9afae32a9ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:07.026660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e5abb-c922-429d-8c82-9afae32a9ab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:07.026945 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e5abb-c922-429d-8c82-9afae32a9ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:07.027273 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e5abb-c922-429d-8c82-9afae32a9ab8 admin admin] PATH_INFO: `/v3/projects/6bff344a6240453595337e67678f4014/users/48eb0c7b55f6433e9d7aad0a91a3aa8a/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:07.038385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bab46-5e6f-4c50-9237-c9e5368bb0c3 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:07.042784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8d352-f186-4aad-8f25-76cf97a8a415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:07.043413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8d352-f186-4aad-8f25-76cf97a8a415 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:07.043784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8d352-f186-4aad-8f25-76cf97a8a415 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:07.044004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8d352-f186-4aad-8f25-76cf97a8a415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:07.047036 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e5abb-c922-429d-8c82-9afae32a9ab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bff344a6240453595337e67678f4014, user_id=48eb0c7b55f6433e9d7aad0a91a3aa8a, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:07.050154 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e5abb-c922-429d-8c82-9afae32a9ab8 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:07.054737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8d352-f186-4aad-8f25-76cf97a8a415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:07.056224 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8d352-f186-4aad-8f25-76cf97a8a415 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:07.069387 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1014/2009] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:06 2022] PUT /identity/v3/projects/6bff344a6240453595337e67678f4014/users/48eb0c7b55f6433e9d7aad0a91a3aa8a/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:07.069574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 996/2010] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:07.069655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e5abb-c922-429d-8c82-9afae32a9ab8 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:07.069730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a9aea-5adf-4090-b39c-d0d2a4a8cd3c None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:07.069730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a9aea-5adf-4090-b39c-d0d2a4a8cd3c None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:07.069730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a9aea-5adf-4090-b39c-d0d2a4a8cd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:07.071991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-3b9e5abb-c922-429d-8c82-9afae32a9ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:07.101516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-4a4a9aea-5adf-4090-b39c-d0d2a4a8cd3c None None] MFA Rules not processed for user `844bec9cbdd54718981c137d31234a57`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:07.102750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4a4a9aea-5adf-4090-b39c-d0d2a4a8cd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:07.147465 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e5abb-c922-429d-8c82-9afae32a9ab8 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:07.151578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7aa28-cb9b-4c99-a8d6-e3ca5a990f09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:07.152369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7aa28-cb9b-4c99-a8d6-e3ca5a990f09 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:07.152708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7aa28-cb9b-4c99-a8d6-e3ca5a990f09 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:07.152988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7aa28-cb9b-4c99-a8d6-e3ca5a990f09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:07.183111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 997/2011] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:07.183885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a9aea-5adf-4090-b39c-d0d2a4a8cd3c None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:07.223277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7aa28-cb9b-4c99-a8d6-e3ca5a990f09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:07.224778 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7aa28-cb9b-4c99-a8d6-e3ca5a990f09 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:07.235343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1015/2012] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:07.236336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7aa28-cb9b-4c99-a8d6-e3ca5a990f09 service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:07.237820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-46d7aa28-cb9b-4c99-a8d6-e3ca5a990f09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:07.266780 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7aa28-cb9b-4c99-a8d6-e3ca5a990f09 service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:07.272230 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f3d33-aa72-434d-a6e2-9a478325c3ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:07.272797 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f3d33-aa72-434d-a6e2-9a478325c3ad service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:07.273078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f3d33-aa72-434d-a6e2-9a478325c3ad service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:07.273620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f3d33-aa72-434d-a6e2-9a478325c3ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:07.282208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a9aea-5adf-4090-b39c-d0d2a4a8cd3c None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:07.285276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f3d33-aa72-434d-a6e2-9a478325c3ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:07.286871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f3d33-aa72-434d-a6e2-9a478325c3ad service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:07.288472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb945969-481e-4f59-b555-bb20947fc3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:07.289123 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb945969-481e-4f59-b555-bb20947fc3da admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:07.289411 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb945969-481e-4f59-b555-bb20947fc3da admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:07.289692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb945969-481e-4f59-b555-bb20947fc3da admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:07.290407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb945969-481e-4f59-b555-bb20947fc3da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:07.292632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb945969-481e-4f59-b555-bb20947fc3da admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:07.298422 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1016/2013] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:07.299159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f3d33-aa72-434d-a6e2-9a478325c3ad service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:07.301061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f22f3d33-aa72-434d-a6e2-9a478325c3ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:07.308864 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 998/2014] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:07.316886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f3d33-aa72-434d-a6e2-9a478325c3ad service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:07.319106 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb945969-481e-4f59-b555-bb20947fc3da admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:07.321741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320f5f2-ec8b-4f15-bb17-dbaec0fae0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:07.322384 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320f5f2-ec8b-4f15-bb17-dbaec0fae0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:07.322663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320f5f2-ec8b-4f15-bb17-dbaec0fae0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:07.322934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320f5f2-ec8b-4f15-bb17-dbaec0fae0d0 admin admin] PATH_INFO: `/v3/users/b1760aa3b3234380b5b092b4966f3d6a` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:07.324143 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320f5f2-ec8b-4f15-bb17-dbaec0fae0d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1760aa3b3234380b5b092b4966f3d6a)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:07.327642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320f5f2-ec8b-4f15-bb17-dbaec0fae0d0 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:07.330635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb945969-481e-4f59-b555-bb20947fc3da admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:07.337633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaad5d4-a715-46c8-94b2-995f7085e660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:07.338361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaad5d4-a715-46c8-94b2-995f7085e660 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:07.338649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaad5d4-a715-46c8-94b2-995f7085e660 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:07.339071 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaad5d4-a715-46c8-94b2-995f7085e660 admin admin] PATH_INFO: `/v3/projects/6bff344a6240453595337e67678f4014/users/48eb0c7b55f6433e9d7aad0a91a3aa8a/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:07.341905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaad5d4-a715-46c8-94b2-995f7085e660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bff344a6240453595337e67678f4014, user_id=48eb0c7b55f6433e9d7aad0a91a3aa8a, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:07.346223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaad5d4-a715-46c8-94b2-995f7085e660 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:07.373243 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 999/2015] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:07 2022] PUT /identity/v3/projects/6bff344a6240453595337e67678f4014/users/48eb0c7b55f6433e9d7aad0a91a3aa8a/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:07.376469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaad5d4-a715-46c8-94b2-995f7085e660 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:07.425251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:07.462544 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-6320f5f2-ec8b-4f15-bb17-dbaec0fae0d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1760aa3b3234380b5b092b4966f3d6a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:07.484760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:07.488377 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-6320f5f2-ec8b-4f15-bb17-dbaec0fae0d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1760aa3b3234380b5b092b4966f3d6a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:07.508719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaad5d4-a715-46c8-94b2-995f7085e660 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:07.514372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230fdbaf-a5ab-4317-9c7e-61382553863d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:07.515285 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230fdbaf-a5ab-4317-9c7e-61382553863d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:07.515406 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-6320f5f2-ec8b-4f15-bb17-dbaec0fae0d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1760aa3b3234380b5b092b4966f3d6a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:07.515847 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230fdbaf-a5ab-4317-9c7e-61382553863d admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:07.516273 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230fdbaf-a5ab-4317-9c7e-61382553863d admin admin] PATH_INFO: `/v3/users/844bec9cbdd54718981c137d31234a57` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:07.517750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230fdbaf-a5ab-4317-9c7e-61382553863d admin admin] RBAC: Authorizing `identity:delete_user(user_id=844bec9cbdd54718981c137d31234a57)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:07.520148 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230fdbaf-a5ab-4317-9c7e-61382553863d admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:07.533224 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-6320f5f2-ec8b-4f15-bb17-dbaec0fae0d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1760aa3b3234380b5b092b4966f3d6a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:07.534599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-6320f5f2-ec8b-4f15-bb17-dbaec0fae0d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1760aa3b3234380b5b092b4966f3d6a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:07.546699 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1017/2016] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:07 2022] DELETE /identity/v3/users/b1760aa3b3234380b5b092b4966f3d6a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:07.547863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfbc16-96e8-49f5-acda-b7b1a4b8fc1c None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:07.548111 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfbc16-96e8-49f5-acda-b7b1a4b8fc1c None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:07.548419 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfbc16-96e8-49f5-acda-b7b1a4b8fc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:07.594178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-230fdbaf-a5ab-4317-9c7e-61382553863d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '844bec9cbdd54718981c137d31234a57'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:07.599830 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-9fbfbc16-96e8-49f5-acda-b7b1a4b8fc1c None None] MFA Rules not processed for user `48eb0c7b55f6433e9d7aad0a91a3aa8a`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:07.601587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-9fbfbc16-96e8-49f5-acda-b7b1a4b8fc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:07.609905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-230fdbaf-a5ab-4317-9c7e-61382553863d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '844bec9cbdd54718981c137d31234a57'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:07.630307 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-230fdbaf-a5ab-4317-9c7e-61382553863d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '844bec9cbdd54718981c137d31234a57'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:07.639966 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-230fdbaf-a5ab-4317-9c7e-61382553863d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '844bec9cbdd54718981c137d31234a57'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:07.642319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-230fdbaf-a5ab-4317-9c7e-61382553863d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '844bec9cbdd54718981c137d31234a57'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:07.653369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1000/2017] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:07 2022] DELETE /identity/v3/users/844bec9cbdd54718981c137d31234a57 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:07.653369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230fdbaf-a5ab-4317-9c7e-61382553863d admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:07.653369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-230fdbaf-a5ab-4317-9c7e-61382553863d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:07.742614 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230fdbaf-a5ab-4317-9c7e-61382553863d admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:07.745576 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec0da0-55a6-4496-9a87-04669fe769a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:07.746348 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec0da0-55a6-4496-9a87-04669fe769a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:07.746697 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1018/2018] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:07 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:07.748351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6187a02-65d7-4ddb-a1b9-7807bba8284d None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:07.748629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6187a02-65d7-4ddb-a1b9-7807bba8284d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:07.748899 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6187a02-65d7-4ddb-a1b9-7807bba8284d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:07.751388 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec0da0-55a6-4496-9a87-04669fe769a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:07.751991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec0da0-55a6-4496-9a87-04669fe769a6 admin admin] PATH_INFO: `/v3/projects/b647c278c63e44148f8bbe0cf8fad86a` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:07.753976 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec0da0-55a6-4496-9a87-04669fe769a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b647c278c63e44148f8bbe0cf8fad86a)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:07.756255 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec0da0-55a6-4496-9a87-04669fe769a6 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:07.771878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d6187a02-65d7-4ddb-a1b9-7807bba8284d None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:07.773631 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d6187a02-65d7-4ddb-a1b9-7807bba8284d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:07.786525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1019/2019] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:07.788668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec39cc-6aef-4211-8066-1e9fffca2153 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:07.789101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec39cc-6aef-4211-8066-1e9fffca2153 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:07.789604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec39cc-6aef-4211-8066-1e9fffca2153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:07.824318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-e6ec39cc-6aef-4211-8066-1e9fffca2153 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:07.824318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e6ec39cc-6aef-4211-8066-1e9fffca2153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:07.824578 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-08ec0da0-55a6-4496-9a87-04669fe769a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b647c278c63e44148f8bbe0cf8fad86a', 'request_id': 'req-08ec0da0-55a6-4496-9a87-04669fe769a6'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:07.830886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-08ec0da0-55a6-4496-9a87-04669fe769a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b647c278c63e44148f8bbe0cf8fad86a', 'request_id': 'req-08ec0da0-55a6-4496-9a87-04669fe769a6'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:07.837680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-08ec0da0-55a6-4496-9a87-04669fe769a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b647c278c63e44148f8bbe0cf8fad86a', 'request_id': 'req-08ec0da0-55a6-4496-9a87-04669fe769a6'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:07.839535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:07.863653 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-08ec0da0-55a6-4496-9a87-04669fe769a6 admin admin] The token cache is being invalidate because project b647c278c63e44148f8bbe0cf8fad86a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:07.864333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-08ec0da0-55a6-4496-9a87-04669fe769a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:07.865894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-08ec0da0-55a6-4496-9a87-04669fe769a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:07.871662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1001/2020] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:07 2022] DELETE /identity/v3/projects/b647c278c63e44148f8bbe0cf8fad86a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:07.874723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec0da0-55a6-4496-9a87-04669fe769a6 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:07.877578 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-08ec0da0-55a6-4496-9a87-04669fe769a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:07.911886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1020/2021] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:07.916023 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec39cc-6aef-4211-8066-1e9fffca2153 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:07.918173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e6ec39cc-6aef-4211-8066-1e9fffca2153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:07.962810 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:07.972136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec0da0-55a6-4496-9a87-04669fe769a6 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:07.976083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930852c7-1cb8-4d85-a83f-8cec06f110b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:07.976932 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930852c7-1cb8-4d85-a83f-8cec06f110b3 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:07.977926 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930852c7-1cb8-4d85-a83f-8cec06f110b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:07.978215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930852c7-1cb8-4d85-a83f-8cec06f110b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:07.980091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-930852c7-1cb8-4d85-a83f-8cec06f110b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:08.021743 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec39cc-6aef-4211-8066-1e9fffca2153 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:08.026885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee176b7-cd37-4336-be45-18086b7d244f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:08.027536 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee176b7-cd37-4336-be45-18086b7d244f service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:08.028505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee176b7-cd37-4336-be45-18086b7d244f service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:08.028839 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee176b7-cd37-4336-be45-18086b7d244f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:08.031770 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2ee176b7-cd37-4336-be45-18086b7d244f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:08.051847 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee176b7-cd37-4336-be45-18086b7d244f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:08.054223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee176b7-cd37-4336-be45-18086b7d244f service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:08.068801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1021/2022] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:08.069552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee176b7-cd37-4336-be45-18086b7d244f service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:08.071278 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2ee176b7-cd37-4336-be45-18086b7d244f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:08.081665 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930852c7-1cb8-4d85-a83f-8cec06f110b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:08.083295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930852c7-1cb8-4d85-a83f-8cec06f110b3 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:08.090633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee176b7-cd37-4336-be45-18086b7d244f service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:08.098292 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1002/2023] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:07 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:08.099602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930852c7-1cb8-4d85-a83f-8cec06f110b3 service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:08.112773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930852c7-1cb8-4d85-a83f-8cec06f110b3 service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:08.114072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2de6c-e1cb-4d98-8599-492d119e65b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:08.114643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2de6c-e1cb-4d98-8599-492d119e65b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:08.114930 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2de6c-e1cb-4d98-8599-492d119e65b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:08.115207 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2de6c-e1cb-4d98-8599-492d119e65b0 admin admin] PATH_INFO: `/v3/users/eb21d5dd2412459ea6ccaa9d707e6b51` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:08.116961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2de6c-e1cb-4d98-8599-492d119e65b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb21d5dd2412459ea6ccaa9d707e6b51)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:08.119606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4ab8b-8d0d-4d4e-9855-e3340672784a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:08.120965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4ab8b-8d0d-4d4e-9855-e3340672784a service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:08.121306 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4ab8b-8d0d-4d4e-9855-e3340672784a service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:08.121618 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2de6c-e1cb-4d98-8599-492d119e65b0 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:08.121749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4ab8b-8d0d-4d4e-9855-e3340672784a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:08.135111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4ab8b-8d0d-4d4e-9855-e3340672784a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:08.136836 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4ab8b-8d0d-4d4e-9855-e3340672784a service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:08.153115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1003/2024] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:08.243305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-58a2de6c-e1cb-4d98-8599-492d119e65b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb21d5dd2412459ea6ccaa9d707e6b51'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:08.259859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-58a2de6c-e1cb-4d98-8599-492d119e65b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb21d5dd2412459ea6ccaa9d707e6b51'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:08.292015 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-58a2de6c-e1cb-4d98-8599-492d119e65b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb21d5dd2412459ea6ccaa9d707e6b51'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:08.313736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-58a2de6c-e1cb-4d98-8599-492d119e65b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb21d5dd2412459ea6ccaa9d707e6b51'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:08.314991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-58a2de6c-e1cb-4d98-8599-492d119e65b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb21d5dd2412459ea6ccaa9d707e6b51'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:08.324828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1022/2025] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:08 2022] DELETE /identity/v3/users/eb21d5dd2412459ea6ccaa9d707e6b51 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:08.370612 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4ab8b-8d0d-4d4e-9855-e3340672784a service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:08.388958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4ab8b-8d0d-4d4e-9855-e3340672784a service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:08.485152 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f6e3c-42e3-4bb3-8573-cffa8978b4fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:08.485783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f6e3c-42e3-4bb3-8573-cffa8978b4fe service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:08.486128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f6e3c-42e3-4bb3-8573-cffa8978b4fe service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:08.486774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f6e3c-42e3-4bb3-8573-cffa8978b4fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:08.488790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-569f6e3c-42e3-4bb3-8573-cffa8978b4fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:08.557223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:08.580508 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f6e3c-42e3-4bb3-8573-cffa8978b4fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:08.582196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f6e3c-42e3-4bb3-8573-cffa8978b4fe service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:08.588046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2de6c-e1cb-4d98-8599-492d119e65b0 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:08.589719 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-58a2de6c-e1cb-4d98-8599-492d119e65b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:08.612402 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1004/2026] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:08 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:08.744619 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f6e3c-42e3-4bb3-8573-cffa8978b4fe service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:08.750583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-569f6e3c-42e3-4bb3-8573-cffa8978b4fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:08.784623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f6e3c-42e3-4bb3-8573-cffa8978b4fe service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:08.794463 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f942a0-bd7a-45aa-9708-8a1e3bd6c533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:08.797459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f942a0-bd7a-45aa-9708-8a1e3bd6c533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:08.797769 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f942a0-bd7a-45aa-9708-8a1e3bd6c533 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:08.798055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f942a0-bd7a-45aa-9708-8a1e3bd6c533 admin admin] PATH_INFO: `/v3/projects/5b9b816a4b694b83a87dbaf44caee57d` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:08.800674 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f942a0-bd7a-45aa-9708-8a1e3bd6c533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b9b816a4b694b83a87dbaf44caee57d)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:08.805253 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f942a0-bd7a-45aa-9708-8a1e3bd6c533 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:08.805339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2de6c-e1cb-4d98-8599-492d119e65b0 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:08.819965 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bc64c-f7a5-4038-8f97-41302e4f331a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:08.821243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bc64c-f7a5-4038-8f97-41302e4f331a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:08.821535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bc64c-f7a5-4038-8f97-41302e4f331a admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:08.821817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bc64c-f7a5-4038-8f97-41302e4f331a admin admin] PATH_INFO: `/v3/projects/471e02957ac348929a4da4710b279883` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:08.826572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bc64c-f7a5-4038-8f97-41302e4f331a admin admin] RBAC: Authorizing `identity:delete_project(project_id=471e02957ac348929a4da4710b279883)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:08.832321 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bc64c-f7a5-4038-8f97-41302e4f331a admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:08.917632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-74f942a0-bd7a-45aa-9708-8a1e3bd6c533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b9b816a4b694b83a87dbaf44caee57d', 'request_id': 'req-74f942a0-bd7a-45aa-9708-8a1e3bd6c533'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:08.921348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-d50bc64c-f7a5-4038-8f97-41302e4f331a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '471e02957ac348929a4da4710b279883', 'request_id': 'req-d50bc64c-f7a5-4038-8f97-41302e4f331a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:08.923188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:08.931026 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-74f942a0-bd7a-45aa-9708-8a1e3bd6c533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b9b816a4b694b83a87dbaf44caee57d', 'request_id': 'req-74f942a0-bd7a-45aa-9708-8a1e3bd6c533'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:08.941503 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-d50bc64c-f7a5-4038-8f97-41302e4f331a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '471e02957ac348929a4da4710b279883', 'request_id': 'req-d50bc64c-f7a5-4038-8f97-41302e4f331a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:08.945441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-74f942a0-bd7a-45aa-9708-8a1e3bd6c533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b9b816a4b694b83a87dbaf44caee57d', 'request_id': 'req-74f942a0-bd7a-45aa-9708-8a1e3bd6c533'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:08.954862 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-d50bc64c-f7a5-4038-8f97-41302e4f331a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '471e02957ac348929a4da4710b279883', 'request_id': 'req-d50bc64c-f7a5-4038-8f97-41302e4f331a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:08.968328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-74f942a0-bd7a-45aa-9708-8a1e3bd6c533 admin admin] The token cache is being invalidate because project 5b9b816a4b694b83a87dbaf44caee57d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:08.968656 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-74f942a0-bd7a-45aa-9708-8a1e3bd6c533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:08.969687 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-74f942a0-bd7a-45aa-9708-8a1e3bd6c533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:08.971301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1005/2027] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:08 2022] DELETE /identity/v3/projects/5b9b816a4b694b83a87dbaf44caee57d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:08.976523 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-d50bc64c-f7a5-4038-8f97-41302e4f331a admin admin] The token cache is being invalidate because project 471e02957ac348929a4da4710b279883 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:08.978731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb01972-54ae-4371-95e7-06fe48f1865c None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:08.979126 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb01972-54ae-4371-95e7-06fe48f1865c None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:08.980200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb01972-54ae-4371-95e7-06fe48f1865c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:08.983620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-d50bc64c-f7a5-4038-8f97-41302e4f331a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:08.984650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-d50bc64c-f7a5-4038-8f97-41302e4f331a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:08.988421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1023/2028] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:08 2022] DELETE /identity/v3/projects/471e02957ac348929a4da4710b279883 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:08.993626 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1734c12-9847-4c29-9b4c-e72b1224c8d2 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:08.995245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1734c12-9847-4c29-9b4c-e72b1224c8d2 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:08.995566 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1734c12-9847-4c29-9b4c-e72b1224c8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.024596 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-9bb01972-54ae-4371-95e7-06fe48f1865c None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:09.026814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9bb01972-54ae-4371-95e7-06fe48f1865c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:09.027053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-c1734c12-9847-4c29-9b4c-e72b1224c8d2 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:09.028579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c1734c12-9847-4c29-9b4c-e72b1224c8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:09.143565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1024/2029] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:09.144593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1734c12-9847-4c29-9b4c-e72b1224c8d2 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.149406 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c1734c12-9847-4c29-9b4c-e72b1224c8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:09.170113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1006/2030] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:09.174039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1734c12-9847-4c29-9b4c-e72b1224c8d2 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.174214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237c039-ca71-40e3-a2ba-439be285a723 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.174307 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237c039-ca71-40e3-a2ba-439be285a723 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.174557 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237c039-ca71-40e3-a2ba-439be285a723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.183221 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32d257-912d-40e7-a19f-79b85068264c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.183961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32d257-912d-40e7-a19f-79b85068264c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.184348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32d257-912d-40e7-a19f-79b85068264c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.184739 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32d257-912d-40e7-a19f-79b85068264c admin admin] PATH_INFO: `/v3/projects/1685a87fa202447996decc9c3371d22d` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.189000 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32d257-912d-40e7-a19f-79b85068264c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1685a87fa202447996decc9c3371d22d)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.192798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32d257-912d-40e7-a19f-79b85068264c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.218403 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-9237c039-ca71-40e3-a2ba-439be285a723 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:09.219923 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9237c039-ca71-40e3-a2ba-439be285a723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:09.229831 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1007/2031] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:09.230975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87228ffc-c629-40fa-bf82-1efb3d54be21 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.231296 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87228ffc-c629-40fa-bf82-1efb3d54be21 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.231649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87228ffc-c629-40fa-bf82-1efb3d54be21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.257544 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-87228ffc-c629-40fa-bf82-1efb3d54be21 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:09.258684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-87228ffc-c629-40fa-bf82-1efb3d54be21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:09.269555 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ea32d257-912d-40e7-a19f-79b85068264c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1685a87fa202447996decc9c3371d22d', 'request_id': 'req-ea32d257-912d-40e7-a19f-79b85068264c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:09.275678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ea32d257-912d-40e7-a19f-79b85068264c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1685a87fa202447996decc9c3371d22d', 'request_id': 'req-ea32d257-912d-40e7-a19f-79b85068264c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:09.284895 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ea32d257-912d-40e7-a19f-79b85068264c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1685a87fa202447996decc9c3371d22d', 'request_id': 'req-ea32d257-912d-40e7-a19f-79b85068264c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:09.299265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ea32d257-912d-40e7-a19f-79b85068264c admin admin] The token cache is being invalidate because project 1685a87fa202447996decc9c3371d22d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:09.299470 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ea32d257-912d-40e7-a19f-79b85068264c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:09.300105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ea32d257-912d-40e7-a19f-79b85068264c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:09.301736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1025/2032] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:09 2022] DELETE /identity/v3/projects/1685a87fa202447996decc9c3371d22d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:09.305370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32d257-912d-40e7-a19f-79b85068264c admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.305370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ea32d257-912d-40e7-a19f-79b85068264c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:09.321362 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:09.336805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1008/2033] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:09.338163 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f492e6-5f2a-4c82-8949-67147383c63d None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.338446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f492e6-5f2a-4c82-8949-67147383c63d None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.338788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f492e6-5f2a-4c82-8949-67147383c63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.365442 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-20f492e6-5f2a-4c82-8949-67147383c63d None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:09.366863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-20f492e6-5f2a-4c82-8949-67147383c63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:09.376316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1009/2034] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:09.377418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f492e6-5f2a-4c82-8949-67147383c63d None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.388892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f492e6-5f2a-4c82-8949-67147383c63d None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.393839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d4fc1-a787-4421-a76a-916d186ac64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.394599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d4fc1-a787-4421-a76a-916d186ac64f admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.394883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d4fc1-a787-4421-a76a-916d186ac64f admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.395226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d4fc1-a787-4421-a76a-916d186ac64f admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.396025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d4fc1-a787-4421-a76a-916d186ac64f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:51:09.396822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d4fc1-a787-4421-a76a-916d186ac64f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.398137 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d4fc1-a787-4421-a76a-916d186ac64f admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.400567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32d257-912d-40e7-a19f-79b85068264c admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.407955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b75c4-e95c-491f-9465-6d611dde9f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.408623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b75c4-e95c-491f-9465-6d611dde9f84 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.408912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b75c4-e95c-491f-9465-6d611dde9f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.409191 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b75c4-e95c-491f-9465-6d611dde9f84 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.409911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b75c4-e95c-491f-9465-6d611dde9f84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:51:09.410384 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b75c4-e95c-491f-9465-6d611dde9f84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.412032 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b75c4-e95c-491f-9465-6d611dde9f84 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.414171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1010/2035] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:51:09.415073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039a9b1-00a7-4734-9038-7799b99627ee None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.415366 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039a9b1-00a7-4734-9038-7799b99627ee None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.417543 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039a9b1-00a7-4734-9038-7799b99627ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.417759 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:09.441855 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-8039a9b1-00a7-4734-9038-7799b99627ee None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:09.442651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8039a9b1-00a7-4734-9038-7799b99627ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:09.442858 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1026/2036] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:51:09.443736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b75c4-e95c-491f-9465-6d611dde9f84 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.451613 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1011/2037] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:09.455466 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039a9b1-00a7-4734-9038-7799b99627ee None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.468500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b75c4-e95c-491f-9465-6d611dde9f84 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.469431 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039a9b1-00a7-4734-9038-7799b99627ee None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.475868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1660b-0398-4f93-becb-36945a1a570c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.476203 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e16e2-365d-4bcf-a513-d3a7cdcd400d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.476610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1660b-0398-4f93-becb-36945a1a570c admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.476897 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1660b-0398-4f93-becb-36945a1a570c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.477201 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1660b-0398-4f93-becb-36945a1a570c admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.477843 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e16e2-365d-4bcf-a513-d3a7cdcd400d admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.477979 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1660b-0398-4f93-becb-36945a1a570c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.478469 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e16e2-365d-4bcf-a513-d3a7cdcd400d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.478839 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e16e2-365d-4bcf-a513-d3a7cdcd400d admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.479696 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e16e2-365d-4bcf-a513-d3a7cdcd400d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.479812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1660b-0398-4f93-becb-36945a1a570c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.483629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e16e2-365d-4bcf-a513-d3a7cdcd400d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.515065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1012/2038] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:09 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:09.516449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1660b-0398-4f93-becb-36945a1a570c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.523196 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1027/2039] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:09 2022] POST /identity/v3/projects => generated 375 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:09.525001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e16e2-365d-4bcf-a513-d3a7cdcd400d admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.606220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1660b-0398-4f93-becb-36945a1a570c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.608054 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e16e2-365d-4bcf-a513-d3a7cdcd400d admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.611843 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d22c1-d579-405c-ba0a-f28f592a6670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.612741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d22c1-d579-405c-ba0a-f28f592a6670 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.613161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d22c1-d579-405c-ba0a-f28f592a6670 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.613476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f213e8a-2425-4c34-a421-c54b991640db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.613606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d22c1-d579-405c-ba0a-f28f592a6670 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.614112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f213e8a-2425-4c34-a421-c54b991640db admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.614240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d22c1-d579-405c-ba0a-f28f592a6670 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:51:09.614392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f213e8a-2425-4c34-a421-c54b991640db admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.614542 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d22c1-d579-405c-ba0a-f28f592a6670 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.614694 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f213e8a-2425-4c34-a421-c54b991640db admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.615383 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f213e8a-2425-4c34-a421-c54b991640db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.616147 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d22c1-d579-405c-ba0a-f28f592a6670 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.622663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f213e8a-2425-4c34-a421-c54b991640db admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.629458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1013/2040] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:51:09.630287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d22c1-d579-405c-ba0a-f28f592a6670 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.642144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d22c1-d579-405c-ba0a-f28f592a6670 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.649718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1028/2041] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:09 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:09.649718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f213e8a-2425-4c34-a421-c54b991640db admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.649969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64788f4f-36fe-4af5-907a-f30c5b848c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.649969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64788f4f-36fe-4af5-907a-f30c5b848c53 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.649969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64788f4f-36fe-4af5-907a-f30c5b848c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.649969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64788f4f-36fe-4af5-907a-f30c5b848c53 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.650297 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64788f4f-36fe-4af5-907a-f30c5b848c53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.651482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64788f4f-36fe-4af5-907a-f30c5b848c53 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.659020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f213e8a-2425-4c34-a421-c54b991640db admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.665124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67320d2-cc6e-44a8-a8d3-c911640e7911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.665740 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67320d2-cc6e-44a8-a8d3-c911640e7911 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.666126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67320d2-cc6e-44a8-a8d3-c911640e7911 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.666272 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67320d2-cc6e-44a8-a8d3-c911640e7911 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.666938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67320d2-cc6e-44a8-a8d3-c911640e7911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.668886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67320d2-cc6e-44a8-a8d3-c911640e7911 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.674081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1014/2042] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:09 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:09.674921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64788f4f-36fe-4af5-907a-f30c5b848c53 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.686588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64788f4f-36fe-4af5-907a-f30c5b848c53 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.691634 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2b59b-03f9-4024-aa83-ccea12c81b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.692254 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2b59b-03f9-4024-aa83-ccea12c81b3f admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.692570 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2b59b-03f9-4024-aa83-ccea12c81b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.692848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2b59b-03f9-4024-aa83-ccea12c81b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.693550 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2b59b-03f9-4024-aa83-ccea12c81b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.695153 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2b59b-03f9-4024-aa83-ccea12c81b3f admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.701892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1029/2043] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:09 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:09.701892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67320d2-cc6e-44a8-a8d3-c911640e7911 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.707483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1015/2044] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:09.708409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2b59b-03f9-4024-aa83-ccea12c81b3f admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.719595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2b59b-03f9-4024-aa83-ccea12c81b3f admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.770493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67320d2-cc6e-44a8-a8d3-c911640e7911 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.775655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a78ee-f4a9-4fec-a241-258dcfe097db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.776345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a78ee-f4a9-4fec-a241-258dcfe097db admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.776671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a78ee-f4a9-4fec-a241-258dcfe097db admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.776945 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a78ee-f4a9-4fec-a241-258dcfe097db admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.777674 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a78ee-f4a9-4fec-a241-258dcfe097db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.779950 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a78ee-f4a9-4fec-a241-258dcfe097db admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.781322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a479e-86ef-42db-891e-f5142914f13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.781955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a479e-86ef-42db-891e-f5142914f13d admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.782259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a479e-86ef-42db-891e-f5142914f13d admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.782518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a479e-86ef-42db-891e-f5142914f13d admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.783214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a479e-86ef-42db-891e-f5142914f13d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.785268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a479e-86ef-42db-891e-f5142914f13d admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.793589 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1030/2045] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:09.793589 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a78ee-f4a9-4fec-a241-258dcfe097db admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.806585 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a78ee-f4a9-4fec-a241-258dcfe097db admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.812182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fda98-91c5-4577-8be3-192802581871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.812955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fda98-91c5-4577-8be3-192802581871 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.813359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fda98-91c5-4577-8be3-192802581871 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.813748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fda98-91c5-4577-8be3-192802581871 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.817706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1016/2046] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:09 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:09.817840 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1fda98-91c5-4577-8be3-192802581871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.817919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a479e-86ef-42db-891e-f5142914f13d admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.817991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1fda98-91c5-4577-8be3-192802581871 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.828734 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a479e-86ef-42db-891e-f5142914f13d admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.828858 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1031/2047] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:09.829384 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fda98-91c5-4577-8be3-192802581871 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.833386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108232d-ad42-4a2e-b416-84944583e722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.834152 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108232d-ad42-4a2e-b416-84944583e722 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.834444 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108232d-ad42-4a2e-b416-84944583e722 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.834739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108232d-ad42-4a2e-b416-84944583e722 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.835445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1108232d-ad42-4a2e-b416-84944583e722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.837074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1108232d-ad42-4a2e-b416-84944583e722 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.838668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:09.840403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fda98-91c5-4577-8be3-192802581871 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.854207 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01692432-3472-438a-8b57-bbeb56f8066d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.854835 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01692432-3472-438a-8b57-bbeb56f8066d admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.855184 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01692432-3472-438a-8b57-bbeb56f8066d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.855455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01692432-3472-438a-8b57-bbeb56f8066d admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.855985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1017/2048] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:09.856153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01692432-3472-438a-8b57-bbeb56f8066d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.856774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108232d-ad42-4a2e-b416-84944583e722 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.857875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01692432-3472-438a-8b57-bbeb56f8066d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.867892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108232d-ad42-4a2e-b416-84944583e722 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.870562 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1032/2049] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:09.871397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01692432-3472-438a-8b57-bbeb56f8066d admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.876204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec1669-a02b-42db-a4ab-2abd6499f367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.877185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec1669-a02b-42db-a4ab-2abd6499f367 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.877549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec1669-a02b-42db-a4ab-2abd6499f367 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.877903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec1669-a02b-42db-a4ab-2abd6499f367 admin admin] PATH_INFO: `/v3/projects/644040693bfd4c19a5cb8884dce08fec/users/9f5f0d5d0ad746fe9f2c8a3af62d6339/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.882643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01692432-3472-438a-8b57-bbeb56f8066d admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:09.887547 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d2c3b-8969-4e5f-a2f2-09b4c2b8fbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:09.889066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d2c3b-8969-4e5f-a2f2-09b4c2b8fbdb admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.889417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d2c3b-8969-4e5f-a2f2-09b4c2b8fbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.889863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d2c3b-8969-4e5f-a2f2-09b4c2b8fbdb admin admin] PATH_INFO: `/v3/projects/f15b6902ce664c0ba8fc4cb0868fdaa3/users/f09070fbc0dc4486ae03bae2701fe85c/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.898368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec1669-a02b-42db-a4ab-2abd6499f367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=644040693bfd4c19a5cb8884dce08fec, user_id=9f5f0d5d0ad746fe9f2c8a3af62d6339, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.901251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec1669-a02b-42db-a4ab-2abd6499f367 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.904212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:09.908363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d2c3b-8969-4e5f-a2f2-09b4c2b8fbdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f15b6902ce664c0ba8fc4cb0868fdaa3, user_id=f09070fbc0dc4486ae03bae2701fe85c, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:09.916336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d2c3b-8969-4e5f-a2f2-09b4c2b8fbdb admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:09.922812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1018/2050] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:09 2022] PUT /identity/v3/projects/644040693bfd4c19a5cb8884dce08fec/users/9f5f0d5d0ad746fe9f2c8a3af62d6339/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:09.923863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec1669-a02b-42db-a4ab-2abd6499f367 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:09.943475 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1033/2051] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:09 2022] PUT /identity/v3/projects/f15b6902ce664c0ba8fc4cb0868fdaa3/users/f09070fbc0dc4486ae03bae2701fe85c/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:09.945200 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c8cbb-fac9-4db1-8c5d-7260024045a6 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:09.945200 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c8cbb-fac9-4db1-8c5d-7260024045a6 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:09.945442 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c8cbb-fac9-4db1-8c5d-7260024045a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:09.979548 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-2f5c8cbb-fac9-4db1-8c5d-7260024045a6 None None] MFA Rules not processed for user `9f5f0d5d0ad746fe9f2c8a3af62d6339`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:09.980838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2f5c8cbb-fac9-4db1-8c5d-7260024045a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:10.006827 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec1669-a02b-42db-a4ab-2abd6499f367 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:10.061356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1034/2052] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:10.062773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5658986-8cf1-4d86-af42-85b418d54e74 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:10.062977 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5658986-8cf1-4d86-af42-85b418d54e74 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:10.063255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5658986-8cf1-4d86-af42-85b418d54e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:10.069349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8f0d7-14b8-4860-a62d-9a5478c4fac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:10.070084 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8f0d7-14b8-4860-a62d-9a5478c4fac6 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:10.070588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8f0d7-14b8-4860-a62d-9a5478c4fac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:10.070955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8f0d7-14b8-4860-a62d-9a5478c4fac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:10.072645 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8f0d7-14b8-4860-a62d-9a5478c4fac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:10.074409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8f0d7-14b8-4860-a62d-9a5478c4fac6 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:10.092031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1019/2053] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:10.093557 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8f0d7-14b8-4860-a62d-9a5478c4fac6 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:10.095430 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-85d8f0d7-14b8-4860-a62d-9a5478c4fac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:10.095750 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-a5658986-8cf1-4d86-af42-85b418d54e74 None None] MFA Rules not processed for user `f09070fbc0dc4486ae03bae2701fe85c`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:10.097068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a5658986-8cf1-4d86-af42-85b418d54e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:10.171588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8f0d7-14b8-4860-a62d-9a5478c4fac6 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:10.176271 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f70df3-4acf-4f2b-a041-840a0467a691 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:10.177229 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f70df3-4acf-4f2b-a041-840a0467a691 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:10.177653 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f70df3-4acf-4f2b-a041-840a0467a691 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:10.178066 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f70df3-4acf-4f2b-a041-840a0467a691 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:10.190374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1035/2054] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:10.190374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5658986-8cf1-4d86-af42-85b418d54e74 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:10.195452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f70df3-4acf-4f2b-a041-840a0467a691 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:10.197141 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f70df3-4acf-4f2b-a041-840a0467a691 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:10.209351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1020/2055] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:10.210508 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f70df3-4acf-4f2b-a041-840a0467a691 service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:10.212368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-66f70df3-4acf-4f2b-a041-840a0467a691 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:10.213044 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5658986-8cf1-4d86-af42-85b418d54e74 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:10.218273 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192b784-9f2a-4b0e-b465-b6379d2bfc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:10.219033 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192b784-9f2a-4b0e-b465-b6379d2bfc69 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:10.219440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192b784-9f2a-4b0e-b465-b6379d2bfc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:10.219791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192b784-9f2a-4b0e-b465-b6379d2bfc69 admin admin] PATH_INFO: `/v3/projects/035822abc4514037b6c8dd42fde84db8/users/72d423641d9247c6ad1870d2ea5d7c98/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:10.226632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f70df3-4acf-4f2b-a041-840a0467a691 service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:10.235637 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cf509-6581-41d0-b457-a869d331e381 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:10.236694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf509-6581-41d0-b457-a869d331e381 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:10.236974 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf509-6581-41d0-b457-a869d331e381 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:10.237248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf509-6581-41d0-b457-a869d331e381 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:10.249917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192b784-9f2a-4b0e-b465-b6379d2bfc69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=035822abc4514037b6c8dd42fde84db8, user_id=72d423641d9247c6ad1870d2ea5d7c98, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:10.254704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192b784-9f2a-4b0e-b465-b6379d2bfc69 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:10.260326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cf509-6581-41d0-b457-a869d331e381 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:10.261273 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cf509-6581-41d0-b457-a869d331e381 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:10.274526 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1021/2056] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:10.280440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1036/2057] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:10 2022] PUT /identity/v3/projects/035822abc4514037b6c8dd42fde84db8/users/72d423641d9247c6ad1870d2ea5d7c98/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:10.293005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343abc1-2e12-4bfe-9a7f-24d9d2dbba18 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:10.293299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343abc1-2e12-4bfe-9a7f-24d9d2dbba18 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:10.293583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343abc1-2e12-4bfe-9a7f-24d9d2dbba18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:10.303295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:10.330709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f343abc1-2e12-4bfe-9a7f-24d9d2dbba18 None None] MFA Rules not processed for user `72d423641d9247c6ad1870d2ea5d7c98`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:10.332432 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f343abc1-2e12-4bfe-9a7f-24d9d2dbba18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:10.452463 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1022/2058] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:10.608051 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192b784-9f2a-4b0e-b465-b6379d2bfc69 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:10.678815 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:10.688487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192b784-9f2a-4b0e-b465-b6379d2bfc69 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:10.694581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfaa07-c57e-4507-b79a-86002fe232ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:10.695768 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfaa07-c57e-4507-b79a-86002fe232ee service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:10.696386 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfaa07-c57e-4507-b79a-86002fe232ee service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:10.696852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfaa07-c57e-4507-b79a-86002fe232ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:10.712188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dfaa07-c57e-4507-b79a-86002fe232ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:10.714030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dfaa07-c57e-4507-b79a-86002fe232ee service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:10.726033 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1037/2059] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:17.359414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24031322-3e91-48e7-96fa-05ad01500c6e None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:17.359955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24031322-3e91-48e7-96fa-05ad01500c6e None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:17.359955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24031322-3e91-48e7-96fa-05ad01500c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:17.380406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-24031322-3e91-48e7-96fa-05ad01500c6e None None] MFA Rules not processed for user `f09070fbc0dc4486ae03bae2701fe85c`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:17.381598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-24031322-3e91-48e7-96fa-05ad01500c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:17.439846 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1023/2060] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:20.720626 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=5 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:24.927651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfaa07-c57e-4507-b79a-86002fe232ee service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:25.074847 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfaa07-c57e-4507-b79a-86002fe232ee service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:25.082199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b18af5-7041-4164-a61c-ae34295ca483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:25.082508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b18af5-7041-4164-a61c-ae34295ca483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:25.084815 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b18af5-7041-4164-a61c-ae34295ca483 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:25.085104 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b18af5-7041-4164-a61c-ae34295ca483 admin admin] PATH_INFO: `/v3/users/72d423641d9247c6ad1870d2ea5d7c98` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:25.087632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b18af5-7041-4164-a61c-ae34295ca483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72d423641d9247c6ad1870d2ea5d7c98)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:25.089772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b18af5-7041-4164-a61c-ae34295ca483 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:25.146937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c6b18af5-7041-4164-a61c-ae34295ca483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72d423641d9247c6ad1870d2ea5d7c98'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:25.155368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c6b18af5-7041-4164-a61c-ae34295ca483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72d423641d9247c6ad1870d2ea5d7c98'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:25.169532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c6b18af5-7041-4164-a61c-ae34295ca483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72d423641d9247c6ad1870d2ea5d7c98'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:25.178776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c6b18af5-7041-4164-a61c-ae34295ca483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72d423641d9247c6ad1870d2ea5d7c98'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:25.181412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c6b18af5-7041-4164-a61c-ae34295ca483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72d423641d9247c6ad1870d2ea5d7c98'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:25.200405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1038/2061] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:24 2022] DELETE /identity/v3/users/72d423641d9247c6ad1870d2ea5d7c98 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:25.201487 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed36a9d-4e5e-4dc0-ba13-bd31f55002d8 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:25.201487 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed36a9d-4e5e-4dc0-ba13-bd31f55002d8 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:25.201487 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed36a9d-4e5e-4dc0-ba13-bd31f55002d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:25.221068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-0ed36a9d-4e5e-4dc0-ba13-bd31f55002d8 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:25.222304 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-0ed36a9d-4e5e-4dc0-ba13-bd31f55002d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:25.283715 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1024/2062] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:25.295180 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b18af5-7041-4164-a61c-ae34295ca483 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:25.296991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c6b18af5-7041-4164-a61c-ae34295ca483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:25.377195 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b18af5-7041-4164-a61c-ae34295ca483 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:25.381491 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a372e5-fa56-45f3-8da2-94ec2c47ca82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:25.382159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a372e5-fa56-45f3-8da2-94ec2c47ca82 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:25.382441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a372e5-fa56-45f3-8da2-94ec2c47ca82 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:25.382764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a372e5-fa56-45f3-8da2-94ec2c47ca82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:25.393431 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed36a9d-4e5e-4dc0-ba13-bd31f55002d8 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:25.394838 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-0ed36a9d-4e5e-4dc0-ba13-bd31f55002d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:25.397149 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a372e5-fa56-45f3-8da2-94ec2c47ca82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:25.398874 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a372e5-fa56-45f3-8da2-94ec2c47ca82 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:25.408535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed36a9d-4e5e-4dc0-ba13-bd31f55002d8 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:25.413343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71547066-7758-4c8d-9a68-88f4906c4a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:25.413705 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1039/2063] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:25.413998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71547066-7758-4c8d-9a68-88f4906c4a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:25.414281 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71547066-7758-4c8d-9a68-88f4906c4a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:25.414559 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71547066-7758-4c8d-9a68-88f4906c4a25 admin admin] PATH_INFO: `/v3/users/2a1bd85a6f8540129a4e3ebd096c64b2` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:25.415740 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71547066-7758-4c8d-9a68-88f4906c4a25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a1bd85a6f8540129a4e3ebd096c64b2)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:25.417718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71547066-7758-4c8d-9a68-88f4906c4a25 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:25.471483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:25.485075 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-71547066-7758-4c8d-9a68-88f4906c4a25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a1bd85a6f8540129a4e3ebd096c64b2'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:25.497648 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-71547066-7758-4c8d-9a68-88f4906c4a25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a1bd85a6f8540129a4e3ebd096c64b2'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:25.512258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-71547066-7758-4c8d-9a68-88f4906c4a25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a1bd85a6f8540129a4e3ebd096c64b2'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:25.522364 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-71547066-7758-4c8d-9a68-88f4906c4a25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a1bd85a6f8540129a4e3ebd096c64b2'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:25.523681 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-71547066-7758-4c8d-9a68-88f4906c4a25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a1bd85a6f8540129a4e3ebd096c64b2'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:25.530751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1025/2064] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:25 2022] DELETE /identity/v3/users/2a1bd85a6f8540129a4e3ebd096c64b2 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:51:25.537977 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344c663-ffb2-4fde-b468-49487017a2db None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:25.538867 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344c663-ffb2-4fde-b468-49487017a2db None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:25.539329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344c663-ffb2-4fde-b468-49487017a2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:25.590052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-f344c663-ffb2-4fde-b468-49487017a2db None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:25.591368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f344c663-ffb2-4fde-b468-49487017a2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:25.612414 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:25.673090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1040/2065] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:25.698873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71547066-7758-4c8d-9a68-88f4906c4a25 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:25.701947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-71547066-7758-4c8d-9a68-88f4906c4a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:25.798578 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71547066-7758-4c8d-9a68-88f4906c4a25 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:25.802791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df5f69-e511-466f-a0d3-5831df44145b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:25.803919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df5f69-e511-466f-a0d3-5831df44145b service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:25.804534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df5f69-e511-466f-a0d3-5831df44145b service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:25.805069 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df5f69-e511-466f-a0d3-5831df44145b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:25.818476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df5f69-e511-466f-a0d3-5831df44145b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:25.822024 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df5f69-e511-466f-a0d3-5831df44145b service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:25.836602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1026/2066] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:25.972432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344c663-ffb2-4fde-b468-49487017a2db None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:25.974547 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f344c663-ffb2-4fde-b468-49487017a2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:25.991451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344c663-ffb2-4fde-b468-49487017a2db None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:25.998833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdf056-b2f5-4171-95ac-d743ee908c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:26.000207 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdf056-b2f5-4171-95ac-d743ee908c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:26.000612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdf056-b2f5-4171-95ac-d743ee908c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:26.001002 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdf056-b2f5-4171-95ac-d743ee908c33 admin admin] PATH_INFO: `/v3/projects/035822abc4514037b6c8dd42fde84db8` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:26.002434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdf056-b2f5-4171-95ac-d743ee908c33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=035822abc4514037b6c8dd42fde84db8)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:26.006624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdf056-b2f5-4171-95ac-d743ee908c33 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:26.066931 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-07cdf056-b2f5-4171-95ac-d743ee908c33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '035822abc4514037b6c8dd42fde84db8', 'request_id': 'req-07cdf056-b2f5-4171-95ac-d743ee908c33'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:26.072596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-07cdf056-b2f5-4171-95ac-d743ee908c33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '035822abc4514037b6c8dd42fde84db8', 'request_id': 'req-07cdf056-b2f5-4171-95ac-d743ee908c33'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:26.084861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-07cdf056-b2f5-4171-95ac-d743ee908c33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '035822abc4514037b6c8dd42fde84db8', 'request_id': 'req-07cdf056-b2f5-4171-95ac-d743ee908c33'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:26.097955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-07cdf056-b2f5-4171-95ac-d743ee908c33 admin admin] The token cache is being invalidate because project 035822abc4514037b6c8dd42fde84db8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:26.098381 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-07cdf056-b2f5-4171-95ac-d743ee908c33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:26.099045 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-07cdf056-b2f5-4171-95ac-d743ee908c33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:26.100616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1041/2067] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:25 2022] DELETE /identity/v3/projects/035822abc4514037b6c8dd42fde84db8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:26.106258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b7ec1-2a06-44a0-b6e9-0c62999207ce None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:26.106554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b7ec1-2a06-44a0-b6e9-0c62999207ce None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:26.107593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b7ec1-2a06-44a0-b6e9-0c62999207ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:26.130839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-c03b7ec1-2a06-44a0-b6e9-0c62999207ce None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:26.132173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c03b7ec1-2a06-44a0-b6e9-0c62999207ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:26.202801 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1027/2068] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:51:26.210354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3918e37-ed4e-4cfd-83ed-d8f2f770e232 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:26.210769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3918e37-ed4e-4cfd-83ed-d8f2f770e232 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:26.211140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3918e37-ed4e-4cfd-83ed-d8f2f770e232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:26.233898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-c3918e37-ed4e-4cfd-83ed-d8f2f770e232 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:26.235296 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c3918e37-ed4e-4cfd-83ed-d8f2f770e232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:26.244328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1042/2069] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:26.249899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b7ec1-2a06-44a0-b6e9-0c62999207ce None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:26.262458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b7ec1-2a06-44a0-b6e9-0c62999207ce None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:26.276502 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262174f-aed6-4ea1-af0d-d76b1847cd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:26.277251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262174f-aed6-4ea1-af0d-d76b1847cd10 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:26.277820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262174f-aed6-4ea1-af0d-d76b1847cd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:26.277820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262174f-aed6-4ea1-af0d-d76b1847cd10 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:26.278647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262174f-aed6-4ea1-af0d-d76b1847cd10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:51:26.279591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262174f-aed6-4ea1-af0d-d76b1847cd10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:26.281993 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262174f-aed6-4ea1-af0d-d76b1847cd10 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:26.296344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1028/2070] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:51:26.299742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3918e37-ed4e-4cfd-83ed-d8f2f770e232 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:26.310370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3918e37-ed4e-4cfd-83ed-d8f2f770e232 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:26.315294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ef5a6-c529-478a-85ee-e981064f3e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:26.315990 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ef5a6-c529-478a-85ee-e981064f3e0c admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:26.316380 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ef5a6-c529-478a-85ee-e981064f3e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:26.316732 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ef5a6-c529-478a-85ee-e981064f3e0c admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:26.317557 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ef5a6-c529-478a-85ee-e981064f3e0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:26.319575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ef5a6-c529-478a-85ee-e981064f3e0c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:26.364226 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1043/2071] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:26 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:26.365464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262174f-aed6-4ea1-af0d-d76b1847cd10 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:26.376384 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262174f-aed6-4ea1-af0d-d76b1847cd10 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:26.429086 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:26.444965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41040c67-3dd6-4890-8ec3-b0bb19e9be4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:26.445589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41040c67-3dd6-4890-8ec3-b0bb19e9be4e admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:26.445877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41040c67-3dd6-4890-8ec3-b0bb19e9be4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:26.446148 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41040c67-3dd6-4890-8ec3-b0bb19e9be4e admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:26.446876 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41040c67-3dd6-4890-8ec3-b0bb19e9be4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:26.448851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41040c67-3dd6-4890-8ec3-b0bb19e9be4e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:26.474174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1029/2072] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:26 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:26.478034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ef5a6-c529-478a-85ee-e981064f3e0c admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:26.488966 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ef5a6-c529-478a-85ee-e981064f3e0c admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:26.493788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195e537-0217-4323-b57b-4bd5607d80b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:26.494411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195e537-0217-4323-b57b-4bd5607d80b0 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:26.494696 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195e537-0217-4323-b57b-4bd5607d80b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:26.494973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195e537-0217-4323-b57b-4bd5607d80b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:26.495714 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195e537-0217-4323-b57b-4bd5607d80b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:26.497386 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195e537-0217-4323-b57b-4bd5607d80b0 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:26.509428 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1044/2073] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:51:26.514230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41040c67-3dd6-4890-8ec3-b0bb19e9be4e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:26.524169 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41040c67-3dd6-4890-8ec3-b0bb19e9be4e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:26.529105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e2479-e9d3-41c9-bcc3-acdd65f22ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:26.529763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e2479-e9d3-41c9-bcc3-acdd65f22ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:26.530015 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e2479-e9d3-41c9-bcc3-acdd65f22ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:26.530293 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e2479-e9d3-41c9-bcc3-acdd65f22ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:26.531189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e2479-e9d3-41c9-bcc3-acdd65f22ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:26.532857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e2479-e9d3-41c9-bcc3-acdd65f22ca8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:26.543264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1030/2074] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:26.548243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195e537-0217-4323-b57b-4bd5607d80b0 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:26.558148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195e537-0217-4323-b57b-4bd5607d80b0 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:26.564948 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15ee10-d798-40f1-85f8-7dce11b8aef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:26.565634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15ee10-d798-40f1-85f8-7dce11b8aef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:26.565914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15ee10-d798-40f1-85f8-7dce11b8aef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:26.566189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15ee10-d798-40f1-85f8-7dce11b8aef4 admin admin] PATH_INFO: `/v3/projects/57b1d50a5fe247d4a52496f4079211e2/users/5c66617ce8844ccbb40a5f906add4352/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:26.586523 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15ee10-d798-40f1-85f8-7dce11b8aef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57b1d50a5fe247d4a52496f4079211e2, user_id=5c66617ce8844ccbb40a5f906add4352, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:26.590848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e2479-e9d3-41c9-bcc3-acdd65f22ca8 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:26.592704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15ee10-d798-40f1-85f8-7dce11b8aef4 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:26.593408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-705e2479-e9d3-41c9-bcc3-acdd65f22ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:26.616114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1045/2075] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:26 2022] PUT /identity/v3/projects/57b1d50a5fe247d4a52496f4079211e2/users/5c66617ce8844ccbb40a5f906add4352/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:26.626711 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaee5fd-2f28-4b02-a41b-000093b86051 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:26.627020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaee5fd-2f28-4b02-a41b-000093b86051 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:26.627280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaee5fd-2f28-4b02-a41b-000093b86051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:26.627946 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e2479-e9d3-41c9-bcc3-acdd65f22ca8 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:26.667303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-bcaee5fd-2f28-4b02-a41b-000093b86051 None None] MFA Rules not processed for user `5c66617ce8844ccbb40a5f906add4352`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:26.668584 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-bcaee5fd-2f28-4b02-a41b-000093b86051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:26.702803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:26.707871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb085fc-3a6c-4064-b5e0-31468491c7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:26.708642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb085fc-3a6c-4064-b5e0-31468491c7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:26.709005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb085fc-3a6c-4064-b5e0-31468491c7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:26.709358 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb085fc-3a6c-4064-b5e0-31468491c7c1 admin admin] PATH_INFO: `/v3/projects/238d8bcf271a4134b5cf3c83b9e9c87b` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:26.710621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb085fc-3a6c-4064-b5e0-31468491c7c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=238d8bcf271a4134b5cf3c83b9e9c87b)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:26.712829 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb085fc-3a6c-4064-b5e0-31468491c7c1 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:26.752989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1046/2076] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:51:26.752989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaee5fd-2f28-4b02-a41b-000093b86051 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:26.752989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-bcaee5fd-2f28-4b02-a41b-000093b86051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:26.767016 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaee5fd-2f28-4b02-a41b-000093b86051 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:26.784456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9cb085fc-3a6c-4064-b5e0-31468491c7c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '238d8bcf271a4134b5cf3c83b9e9c87b', 'request_id': 'req-9cb085fc-3a6c-4064-b5e0-31468491c7c1'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:26.784456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9cb085fc-3a6c-4064-b5e0-31468491c7c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '238d8bcf271a4134b5cf3c83b9e9c87b', 'request_id': 'req-9cb085fc-3a6c-4064-b5e0-31468491c7c1'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:26.790135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9cb085fc-3a6c-4064-b5e0-31468491c7c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '238d8bcf271a4134b5cf3c83b9e9c87b', 'request_id': 'req-9cb085fc-3a6c-4064-b5e0-31468491c7c1'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:26.804240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9cb085fc-3a6c-4064-b5e0-31468491c7c1 admin admin] The token cache is being invalidate because project 238d8bcf271a4134b5cf3c83b9e9c87b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:26.804574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9cb085fc-3a6c-4064-b5e0-31468491c7c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:26.805226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9cb085fc-3a6c-4064-b5e0-31468491c7c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:26.806821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1031/2077] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:26 2022] DELETE /identity/v3/projects/238d8bcf271a4134b5cf3c83b9e9c87b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:26.810965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb085fc-3a6c-4064-b5e0-31468491c7c1 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:26.812666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9cb085fc-3a6c-4064-b5e0-31468491c7c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:26.828552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ef254-d88e-4c38-8a83-92dbfac4a325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:26.829208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ef254-d88e-4c38-8a83-92dbfac4a325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:26.829499 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ef254-d88e-4c38-8a83-92dbfac4a325 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:26.829772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ef254-d88e-4c38-8a83-92dbfac4a325 admin admin] PATH_INFO: `/v3/users/9f5f0d5d0ad746fe9f2c8a3af62d6339` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:26.831036 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ef254-d88e-4c38-8a83-92dbfac4a325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f5f0d5d0ad746fe9f2c8a3af62d6339)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:26.833063 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ef254-d88e-4c38-8a83-92dbfac4a325 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:26.866107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:26.882888 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb085fc-3a6c-4064-b5e0-31468491c7c1 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:26.884636 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-d19ef254-d88e-4c38-8a83-92dbfac4a325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f5f0d5d0ad746fe9f2c8a3af62d6339'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:26.889301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9d459-c7be-463e-865b-83b0b1695d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:26.890140 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9d459-c7be-463e-865b-83b0b1695d93 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:26.890548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9d459-c7be-463e-865b-83b0b1695d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:26.890861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9d459-c7be-463e-865b-83b0b1695d93 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:26.891633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9d459-c7be-463e-865b-83b0b1695d93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:26.893995 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9d459-c7be-463e-865b-83b0b1695d93 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:26.895867 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-d19ef254-d88e-4c38-8a83-92dbfac4a325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f5f0d5d0ad746fe9f2c8a3af62d6339'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:26.915397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-d19ef254-d88e-4c38-8a83-92dbfac4a325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f5f0d5d0ad746fe9f2c8a3af62d6339'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:26.929649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1032/2078] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:26 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:26.931525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9d459-c7be-463e-865b-83b0b1695d93 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:26.939205 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-baa9d459-c7be-463e-865b-83b0b1695d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:26.940923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-d19ef254-d88e-4c38-8a83-92dbfac4a325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f5f0d5d0ad746fe9f2c8a3af62d6339'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:26.942130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-d19ef254-d88e-4c38-8a83-92dbfac4a325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f5f0d5d0ad746fe9f2c8a3af62d6339'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:26.951478 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1047/2079] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:26 2022] DELETE /identity/v3/users/9f5f0d5d0ad746fe9f2c8a3af62d6339 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:26.952325 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ef254-d88e-4c38-8a83-92dbfac4a325 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:26.955893 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d19ef254-d88e-4c38-8a83-92dbfac4a325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:27.032057 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9d459-c7be-463e-865b-83b0b1695d93 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:27.044939 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:27.060567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ef254-d88e-4c38-8a83-92dbfac4a325 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:27.065560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab120e-afd4-4d0b-a6f1-5f01a95465ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:27.066274 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab120e-afd4-4d0b-a6f1-5f01a95465ac admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:27.066627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab120e-afd4-4d0b-a6f1-5f01a95465ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:27.066904 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab120e-afd4-4d0b-a6f1-5f01a95465ac admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:27.067577 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab120e-afd4-4d0b-a6f1-5f01a95465ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:27.069561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab120e-afd4-4d0b-a6f1-5f01a95465ac admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:27.093376 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1048/2080] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:26 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:27.094772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc1e40-109e-4d8b-9875-e50d5c571035 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:27.095152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc1e40-109e-4d8b-9875-e50d5c571035 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:27.095560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc1e40-109e-4d8b-9875-e50d5c571035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:27.095672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52a456-e09a-467f-9215-316d0b4136b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:27.096321 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52a456-e09a-467f-9215-316d0b4136b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:27.096527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52a456-e09a-467f-9215-316d0b4136b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:27.096838 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52a456-e09a-467f-9215-316d0b4136b2 admin admin] PATH_INFO: `/v3/users/48eb0c7b55f6433e9d7aad0a91a3aa8a` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:27.098076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52a456-e09a-467f-9215-316d0b4136b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48eb0c7b55f6433e9d7aad0a91a3aa8a)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:27.099994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52a456-e09a-467f-9215-316d0b4136b2 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:27.115326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-dbcc1e40-109e-4d8b-9875-e50d5c571035 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:27.116569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-dbcc1e40-109e-4d8b-9875-e50d5c571035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:27.125565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1049/2081] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:27.131041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc1e40-109e-4d8b-9875-e50d5c571035 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:27.143003 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc1e40-109e-4d8b-9875-e50d5c571035 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:27.148848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f0147-dc7a-468a-bbb7-5fdf0ebf2f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:27.149591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f0147-dc7a-468a-bbb7-5fdf0ebf2f57 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:27.149933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f0147-dc7a-468a-bbb7-5fdf0ebf2f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:27.150206 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f0147-dc7a-468a-bbb7-5fdf0ebf2f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:27.150903 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f0147-dc7a-468a-bbb7-5fdf0ebf2f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:27.152632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f0147-dc7a-468a-bbb7-5fdf0ebf2f57 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:27.161731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9c52a456-e09a-467f-9215-316d0b4136b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48eb0c7b55f6433e9d7aad0a91a3aa8a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:27.166732 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1050/2082] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:27.166732 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f0147-dc7a-468a-bbb7-5fdf0ebf2f57 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:27.166732 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-133f0147-dc7a-468a-bbb7-5fdf0ebf2f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:27.171378 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9c52a456-e09a-467f-9215-316d0b4136b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48eb0c7b55f6433e9d7aad0a91a3aa8a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:27.191456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9c52a456-e09a-467f-9215-316d0b4136b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48eb0c7b55f6433e9d7aad0a91a3aa8a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:27.204604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9c52a456-e09a-467f-9215-316d0b4136b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48eb0c7b55f6433e9d7aad0a91a3aa8a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:27.205539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9c52a456-e09a-467f-9215-316d0b4136b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48eb0c7b55f6433e9d7aad0a91a3aa8a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:27.211831 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1033/2083] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:26 2022] DELETE /identity/v3/users/48eb0c7b55f6433e9d7aad0a91a3aa8a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:27.212785 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52a456-e09a-467f-9215-316d0b4136b2 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:27.214304 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9c52a456-e09a-467f-9215-316d0b4136b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:27.220007 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:27.244428 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f0147-dc7a-468a-bbb7-5fdf0ebf2f57 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:27.295822 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52a456-e09a-467f-9215-316d0b4136b2 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:27.306037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bf941-9011-4613-9647-84ffddd99103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:27.306795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bf941-9011-4613-9647-84ffddd99103 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:27.307080 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bf941-9011-4613-9647-84ffddd99103 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:27.307353 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bf941-9011-4613-9647-84ffddd99103 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:27.308365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bf941-9011-4613-9647-84ffddd99103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:27.308485 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fbcb1-a1e5-4d7f-b429-f25f2b1bda38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:27.309093 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fbcb1-a1e5-4d7f-b429-f25f2b1bda38 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:27.309380 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fbcb1-a1e5-4d7f-b429-f25f2b1bda38 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:27.309660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fbcb1-a1e5-4d7f-b429-f25f2b1bda38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:27.312408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bf941-9011-4613-9647-84ffddd99103 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:27.312558 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ed1fbcb1-a1e5-4d7f-b429-f25f2b1bda38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:27.325267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1034/2084] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:27.328088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fbcb1-a1e5-4d7f-b429-f25f2b1bda38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:27.329667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fbcb1-a1e5-4d7f-b429-f25f2b1bda38 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:27.330257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bf941-9011-4613-9647-84ffddd99103 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:27.342272 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1051/2085] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:27.351250 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bf941-9011-4613-9647-84ffddd99103 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:27.356101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d8f80-213a-4aa3-bdc8-c6e32671667d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:27.357060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d8f80-213a-4aa3-bdc8-c6e32671667d admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:27.357199 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d8f80-213a-4aa3-bdc8-c6e32671667d admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:27.357963 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d8f80-213a-4aa3-bdc8-c6e32671667d admin admin] PATH_INFO: `/v3/projects/9688704c47a04b91affe95a40d1a8908/users/c5b28d90873d4f408ec0e5e70f17b583/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:27.381253 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d8f80-213a-4aa3-bdc8-c6e32671667d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9688704c47a04b91affe95a40d1a8908, user_id=c5b28d90873d4f408ec0e5e70f17b583, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:27.384410 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d8f80-213a-4aa3-bdc8-c6e32671667d admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:27.400760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1035/2086] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:27 2022] PUT /identity/v3/projects/9688704c47a04b91affe95a40d1a8908/users/c5b28d90873d4f408ec0e5e70f17b583/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:27.413456 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5809b10-1d6d-4d3d-866d-95f62e93c769 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:27.413814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5809b10-1d6d-4d3d-866d-95f62e93c769 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:27.414034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5809b10-1d6d-4d3d-866d-95f62e93c769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:27.446539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-b5809b10-1d6d-4d3d-866d-95f62e93c769 None None] MFA Rules not processed for user `c5b28d90873d4f408ec0e5e70f17b583`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:27.447904 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b5809b10-1d6d-4d3d-866d-95f62e93c769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:27.546645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1052/2087] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:27.552746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d8f80-213a-4aa3-bdc8-c6e32671667d admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:27.643790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d8f80-213a-4aa3-bdc8-c6e32671667d admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:27.649740 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af0408-f4dc-409d-926b-8282b3eba8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:27.650659 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af0408-f4dc-409d-926b-8282b3eba8ca admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:27.651395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af0408-f4dc-409d-926b-8282b3eba8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:27.651886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af0408-f4dc-409d-926b-8282b3eba8ca admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:27.653019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af0408-f4dc-409d-926b-8282b3eba8ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:27.654891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af0408-f4dc-409d-926b-8282b3eba8ca admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:27.694832 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1036/2088] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:27 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:51:27.698890 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5809b10-1d6d-4d3d-866d-95f62e93c769 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:27.712066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5809b10-1d6d-4d3d-866d-95f62e93c769 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:27.726387 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:27.764476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af0408-f4dc-409d-926b-8282b3eba8ca admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:27.767558 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a9af0408-f4dc-409d-926b-8282b3eba8ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:27.845049 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7986f-c2d9-4878-aded-89a442a3407f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:27.845692 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7986f-c2d9-4878-aded-89a442a3407f admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:27.845975 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7986f-c2d9-4878-aded-89a442a3407f admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:27.846283 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7986f-c2d9-4878-aded-89a442a3407f admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:27.847424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7986f-c2d9-4878-aded-89a442a3407f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:27.849951 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7986f-c2d9-4878-aded-89a442a3407f admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:27.883020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1053/2089] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:27 2022] POST /identity/v3/users => generated 295 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:27.886920 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7986f-c2d9-4878-aded-89a442a3407f admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:27.889459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8fd7986f-c2d9-4878-aded-89a442a3407f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:27.901452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af0408-f4dc-409d-926b-8282b3eba8ca admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:27.902539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7986f-c2d9-4878-aded-89a442a3407f admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:27.908329 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d7a3e-dd6d-49b2-9713-cf0cc97b34bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:27.908920 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067edea-005f-457f-b958-623153bf0159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:27.909195 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d7a3e-dd6d-49b2-9713-cf0cc97b34bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:27.909489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d7a3e-dd6d-49b2-9713-cf0cc97b34bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:27.909756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d7a3e-dd6d-49b2-9713-cf0cc97b34bb admin admin] PATH_INFO: `/v3/projects/6bff344a6240453595337e67678f4014` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:27.910081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067edea-005f-457f-b958-623153bf0159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:27.910458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067edea-005f-457f-b958-623153bf0159 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:27.910833 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067edea-005f-457f-b958-623153bf0159 admin admin] PATH_INFO: `/v3/projects/644040693bfd4c19a5cb8884dce08fec` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:27.910978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d7a3e-dd6d-49b2-9713-cf0cc97b34bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bff344a6240453595337e67678f4014)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:27.912336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067edea-005f-457f-b958-623153bf0159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=644040693bfd4c19a5cb8884dce08fec)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:27.913004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d7a3e-dd6d-49b2-9713-cf0cc97b34bb admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:27.914652 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067edea-005f-457f-b958-623153bf0159 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:27.916581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:27.978863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4067edea-005f-457f-b958-623153bf0159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '644040693bfd4c19a5cb8884dce08fec', 'request_id': 'req-4067edea-005f-457f-b958-623153bf0159'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:27.980238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-761d7a3e-dd6d-49b2-9713-cf0cc97b34bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bff344a6240453595337e67678f4014', 'request_id': 'req-761d7a3e-dd6d-49b2-9713-cf0cc97b34bb'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:27.984984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4067edea-005f-457f-b958-623153bf0159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '644040693bfd4c19a5cb8884dce08fec', 'request_id': 'req-4067edea-005f-457f-b958-623153bf0159'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:27.987093 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-761d7a3e-dd6d-49b2-9713-cf0cc97b34bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bff344a6240453595337e67678f4014', 'request_id': 'req-761d7a3e-dd6d-49b2-9713-cf0cc97b34bb'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:27.991926 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4067edea-005f-457f-b958-623153bf0159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '644040693bfd4c19a5cb8884dce08fec', 'request_id': 'req-4067edea-005f-457f-b958-623153bf0159'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:27.996320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-761d7a3e-dd6d-49b2-9713-cf0cc97b34bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bff344a6240453595337e67678f4014', 'request_id': 'req-761d7a3e-dd6d-49b2-9713-cf0cc97b34bb'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:28.008944 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4067edea-005f-457f-b958-623153bf0159 admin admin] The token cache is being invalidate because project 644040693bfd4c19a5cb8884dce08fec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:28.009352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4067edea-005f-457f-b958-623153bf0159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:28.010189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4067edea-005f-457f-b958-623153bf0159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:28.012087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1037/2090] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:27 2022] DELETE /identity/v3/projects/644040693bfd4c19a5cb8884dce08fec => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:28.012927 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067edea-005f-457f-b958-623153bf0159 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.017281 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-761d7a3e-dd6d-49b2-9713-cf0cc97b34bb admin admin] The token cache is being invalidate because project 6bff344a6240453595337e67678f4014 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:28.017475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4067edea-005f-457f-b958-623153bf0159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:28.017732 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-761d7a3e-dd6d-49b2-9713-cf0cc97b34bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:28.019593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-761d7a3e-dd6d-49b2-9713-cf0cc97b34bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:28.021268 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1054/2091] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:27 2022] DELETE /identity/v3/projects/6bff344a6240453595337e67678f4014 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:28.022723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73270cc-75be-4378-955c-71314cf0f200 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.023061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73270cc-75be-4378-955c-71314cf0f200 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.024622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73270cc-75be-4378-955c-71314cf0f200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.047818 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d73270cc-75be-4378-955c-71314cf0f200 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:28.049346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d73270cc-75be-4378-955c-71314cf0f200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:28.112114 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067edea-005f-457f-b958-623153bf0159 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.116737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a8759-240d-4584-b33c-c67d876d388c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.117335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a8759-240d-4584-b33c-c67d876d388c admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.117613 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a8759-240d-4584-b33c-c67d876d388c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.117885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a8759-240d-4584-b33c-c67d876d388c admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.118592 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a8759-240d-4584-b33c-c67d876d388c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.119298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1055/2092] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:28.120507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8100d2c-f95b-4562-aef7-d806d8f1e1aa None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.120872 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8100d2c-f95b-4562-aef7-d806d8f1e1aa None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.121221 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8100d2c-f95b-4562-aef7-d806d8f1e1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.125487 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a8759-240d-4584-b33c-c67d876d388c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:28.137632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1038/2093] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:28.138929 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea108c-340d-437e-8034-a6ef2d4832b1 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.139631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea108c-340d-437e-8034-a6ef2d4832b1 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.139781 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea108c-340d-437e-8034-a6ef2d4832b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.149152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-e8100d2c-f95b-4562-aef7-d806d8f1e1aa None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:28.152484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e8100d2c-f95b-4562-aef7-d806d8f1e1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:28.162231 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1056/2094] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:28.162940 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8100d2c-f95b-4562-aef7-d806d8f1e1aa None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.163667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-9eea108c-340d-437e-8034-a6ef2d4832b1 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:28.164204 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e8100d2c-f95b-4562-aef7-d806d8f1e1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:28.164940 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9eea108c-340d-437e-8034-a6ef2d4832b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:28.174268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1039/2095] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:28.175644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aed36-24ad-4293-8132-e2ba812439ad None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.175933 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aed36-24ad-4293-8132-e2ba812439ad None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.176211 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aed36-24ad-4293-8132-e2ba812439ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.183640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8100d2c-f95b-4562-aef7-d806d8f1e1aa None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.190981 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22e772-9095-47aa-8489-800a6704b6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.192417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22e772-9095-47aa-8489-800a6704b6df admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.192841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22e772-9095-47aa-8489-800a6704b6df admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.193200 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22e772-9095-47aa-8489-800a6704b6df admin admin] PATH_INFO: `/v3/projects/84f9a168b3cb46569731ca18e08aa742/users/d867d93d59ce4ed78ce24e99671367b0/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.197258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-a18aed36-24ad-4293-8132-e2ba812439ad None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:28.198518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a18aed36-24ad-4293-8132-e2ba812439ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:28.207293 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1040/2096] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:28.208214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aed36-24ad-4293-8132-e2ba812439ad None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.214779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22e772-9095-47aa-8489-800a6704b6df admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f9a168b3cb46569731ca18e08aa742, user_id=d867d93d59ce4ed78ce24e99671367b0, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.218650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22e772-9095-47aa-8489-800a6704b6df admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:28.218790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aed36-24ad-4293-8132-e2ba812439ad None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.222514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:28.236223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b976d7-3438-4d34-9751-aebff44c5982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.236893 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b976d7-3438-4d34-9751-aebff44c5982 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.237175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b976d7-3438-4d34-9751-aebff44c5982 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.237450 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b976d7-3438-4d34-9751-aebff44c5982 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.238166 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b976d7-3438-4d34-9751-aebff44c5982 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:51:28.238470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b976d7-3438-4d34-9751-aebff44c5982 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.239244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1057/2097] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:28 2022] PUT /identity/v3/projects/84f9a168b3cb46569731ca18e08aa742/users/d867d93d59ce4ed78ce24e99671367b0/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:28.240336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22e772-9095-47aa-8489-800a6704b6df admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.243421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b976d7-3438-4d34-9751-aebff44c5982 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:28.257519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1041/2098] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:51:28.261898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b976d7-3438-4d34-9751-aebff44c5982 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.276488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b976d7-3438-4d34-9751-aebff44c5982 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.343552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22e772-9095-47aa-8489-800a6704b6df admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.344131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c905e-eea2-445e-af87-090caeb092b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.344841 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c905e-eea2-445e-af87-090caeb092b1 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.345127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c905e-eea2-445e-af87-090caeb092b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.345561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c905e-eea2-445e-af87-090caeb092b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.346259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c905e-eea2-445e-af87-090caeb092b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.347861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c905e-eea2-445e-af87-090caeb092b1 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:28.349367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:28.353544 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df60202-d1ef-4f91-a6e4-3ecccdf7dc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.354276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df60202-d1ef-4f91-a6e4-3ecccdf7dc06 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.354684 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df60202-d1ef-4f91-a6e4-3ecccdf7dc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.355153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df60202-d1ef-4f91-a6e4-3ecccdf7dc06 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.355888 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df60202-d1ef-4f91-a6e4-3ecccdf7dc06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:51:28.356855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df60202-d1ef-4f91-a6e4-3ecccdf7dc06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.359870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df60202-d1ef-4f91-a6e4-3ecccdf7dc06 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:28.363535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1042/2099] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:28.364491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c905e-eea2-445e-af87-090caeb092b1 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.377454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c905e-eea2-445e-af87-090caeb092b1 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.386784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e787ce4-296a-45ee-a843-a347f210d24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.387573 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e787ce4-296a-45ee-a843-a347f210d24d admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.387920 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e787ce4-296a-45ee-a843-a347f210d24d admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.388801 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e787ce4-296a-45ee-a843-a347f210d24d admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.389702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e787ce4-296a-45ee-a843-a347f210d24d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.400668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e787ce4-296a-45ee-a843-a347f210d24d admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:28.403240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1058/2100] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:51:28.404343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df60202-d1ef-4f91-a6e4-3ecccdf7dc06 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.420959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df60202-d1ef-4f91-a6e4-3ecccdf7dc06 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.429143 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9e063-0928-48de-958e-a7a69da2ddbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.429844 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9e063-0928-48de-958e-a7a69da2ddbd admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.430337 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9e063-0928-48de-958e-a7a69da2ddbd admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.430898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9e063-0928-48de-958e-a7a69da2ddbd admin admin] PATH_INFO: `/v3/projects/84f9a168b3cb46569731ca18e08aa742/users/d867d93d59ce4ed78ce24e99671367b0/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.433297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9e063-0928-48de-958e-a7a69da2ddbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f9a168b3cb46569731ca18e08aa742, user_id=d867d93d59ce4ed78ce24e99671367b0, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.436261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9e063-0928-48de-958e-a7a69da2ddbd admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:28.442372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1043/2101] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:28 2022] POST /identity/v3/projects => generated 369 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:28.443224 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e787ce4-296a-45ee-a843-a347f210d24d admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.454049 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1059/2102] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:28 2022] PUT /identity/v3/projects/84f9a168b3cb46569731ca18e08aa742/users/d867d93d59ce4ed78ce24e99671367b0/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:28.457387 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9e063-0928-48de-958e-a7a69da2ddbd admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.517816 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e787ce4-296a-45ee-a843-a347f210d24d admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.522788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df88eec-22bb-4013-911d-04bb8e3f3ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.523419 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df88eec-22bb-4013-911d-04bb8e3f3ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.523706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df88eec-22bb-4013-911d-04bb8e3f3ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.523984 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df88eec-22bb-4013-911d-04bb8e3f3ef9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.524742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df88eec-22bb-4013-911d-04bb8e3f3ef9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.526861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df88eec-22bb-4013-911d-04bb8e3f3ef9 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:28.528040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9e063-0928-48de-958e-a7a69da2ddbd admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.534073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834fc29-a994-46b7-8982-b0e2dbf92245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.534869 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834fc29-a994-46b7-8982-b0e2dbf92245 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.535154 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834fc29-a994-46b7-8982-b0e2dbf92245 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.535427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834fc29-a994-46b7-8982-b0e2dbf92245 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.536315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834fc29-a994-46b7-8982-b0e2dbf92245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.538255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834fc29-a994-46b7-8982-b0e2dbf92245 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:28.561013 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1044/2103] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:28 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:28.562291 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508428ee-226d-4925-8bbd-38e9f1e70c0d None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.562585 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508428ee-226d-4925-8bbd-38e9f1e70c0d None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.562866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508428ee-226d-4925-8bbd-38e9f1e70c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.566886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1060/2104] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:28 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:28.566886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834fc29-a994-46b7-8982-b0e2dbf92245 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.576519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834fc29-a994-46b7-8982-b0e2dbf92245 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.597115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-508428ee-226d-4925-8bbd-38e9f1e70c0d None None] MFA Rules not processed for user `d867d93d59ce4ed78ce24e99671367b0`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:28.598168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-508428ee-226d-4925-8bbd-38e9f1e70c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:28.640923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eb307-d39a-4b04-9bea-0fba00fa8ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.641995 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eb307-d39a-4b04-9bea-0fba00fa8ce4 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.642192 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eb307-d39a-4b04-9bea-0fba00fa8ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.642476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eb307-d39a-4b04-9bea-0fba00fa8ce4 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.643155 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973eb307-d39a-4b04-9bea-0fba00fa8ce4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.645147 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973eb307-d39a-4b04-9bea-0fba00fa8ce4 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:28.656534 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:28.671976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1061/2105] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:28 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:51:28.671976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eb307-d39a-4b04-9bea-0fba00fa8ce4 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.686972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eb307-d39a-4b04-9bea-0fba00fa8ce4 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.695444 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1045/2106] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:28.695444 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508428ee-226d-4925-8bbd-38e9f1e70c0d None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.697121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21119-926f-432b-999d-bfd265d45b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.698380 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21119-926f-432b-999d-bfd265d45b41 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.699010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21119-926f-432b-999d-bfd265d45b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.699598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21119-926f-432b-999d-bfd265d45b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.700623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b21119-926f-432b-999d-bfd265d45b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.704018 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b21119-926f-432b-999d-bfd265d45b41 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:28.707747 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508428ee-226d-4925-8bbd-38e9f1e70c0d None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.713669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86184e5e-b6b8-4c3f-8620-21bed323dfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.714390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86184e5e-b6b8-4c3f-8620-21bed323dfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.714766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86184e5e-b6b8-4c3f-8620-21bed323dfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.715133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86184e5e-b6b8-4c3f-8620-21bed323dfb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.715912 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86184e5e-b6b8-4c3f-8620-21bed323dfb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.718292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1062/2107] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:28.718410 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86184e5e-b6b8-4c3f-8620-21bed323dfb8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:28.719649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21119-926f-432b-999d-bfd265d45b41 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.723081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-12b21119-926f-432b-999d-bfd265d45b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:28.749761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1046/2108] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:28.754505 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86184e5e-b6b8-4c3f-8620-21bed323dfb8 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.773728 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86184e5e-b6b8-4c3f-8620-21bed323dfb8 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.779349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a401d-4ff3-4d83-b7ac-58eb65c57e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.780265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a401d-4ff3-4d83-b7ac-58eb65c57e88 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.780645 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a401d-4ff3-4d83-b7ac-58eb65c57e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.780991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a401d-4ff3-4d83-b7ac-58eb65c57e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.781830 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a401d-4ff3-4d83-b7ac-58eb65c57e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.783525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a401d-4ff3-4d83-b7ac-58eb65c57e88 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:28.796353 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1047/2109] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:28.797258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a401d-4ff3-4d83-b7ac-58eb65c57e88 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.833879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a401d-4ff3-4d83-b7ac-58eb65c57e88 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.852001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c1749-ec33-4f9b-8405-72366f11242a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.852931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c1749-ec33-4f9b-8405-72366f11242a admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.853260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c1749-ec33-4f9b-8405-72366f11242a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.853499 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c1749-ec33-4f9b-8405-72366f11242a admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.855958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c1749-ec33-4f9b-8405-72366f11242a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.860019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c1749-ec33-4f9b-8405-72366f11242a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:28.886001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1048/2110] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:28.887190 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c1749-ec33-4f9b-8405-72366f11242a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.902203 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21119-926f-432b-999d-bfd265d45b41 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.903725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c1749-ec33-4f9b-8405-72366f11242a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:28.906833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3caac-977f-4782-8fc4-0d70efca5397 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.907477 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3caac-977f-4782-8fc4-0d70efca5397 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.907956 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3caac-977f-4782-8fc4-0d70efca5397 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.908332 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3caac-977f-4782-8fc4-0d70efca5397 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.908772 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e8f99-704e-4e83-a546-21f43cbfa723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:28.909445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e8f99-704e-4e83-a546-21f43cbfa723 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:28.909861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e8f99-704e-4e83-a546-21f43cbfa723 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:28.910145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e8f99-704e-4e83-a546-21f43cbfa723 admin admin] PATH_INFO: `/v3/projects/d283268ddb0c4b7b91dbc1bae271c708/users/ee01546adfef4b2b94a3fc0e47b71edd/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:28.910617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b2d3caac-977f-4782-8fc4-0d70efca5397 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:28.925434 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:28.941123 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e8f99-704e-4e83-a546-21f43cbfa723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d283268ddb0c4b7b91dbc1bae271c708, user_id=ee01546adfef4b2b94a3fc0e47b71edd, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.944880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e8f99-704e-4e83-a546-21f43cbfa723 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:28.960683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1049/2111] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:28 2022] PUT /identity/v3/projects/d283268ddb0c4b7b91dbc1bae271c708/users/ee01546adfef4b2b94a3fc0e47b71edd/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:28.961812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e8f99-704e-4e83-a546-21f43cbfa723 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:28.992412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3caac-977f-4782-8fc4-0d70efca5397 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:28.994372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3caac-977f-4782-8fc4-0d70efca5397 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:29.006101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1063/2112] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:29.007138 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd62b2-9b3d-4cfc-afb4-bdda56dca773 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:29.007444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd62b2-9b3d-4cfc-afb4-bdda56dca773 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:29.007730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd62b2-9b3d-4cfc-afb4-bdda56dca773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:29.038807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-b1dd62b2-9b3d-4cfc-afb4-bdda56dca773 None None] MFA Rules not processed for user `ee01546adfef4b2b94a3fc0e47b71edd`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:29.039622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b1dd62b2-9b3d-4cfc-afb4-bdda56dca773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:29.043452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e8f99-704e-4e83-a546-21f43cbfa723 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:29.051025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f112f5-a26b-4e45-ba50-f41764dda933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:29.052087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f112f5-a26b-4e45-ba50-f41764dda933 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:29.052504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f112f5-a26b-4e45-ba50-f41764dda933 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:29.052941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f112f5-a26b-4e45-ba50-f41764dda933 admin admin] PATH_INFO: `/v3/projects/4b7336552c874aa9a766288070767ae3/users/7975c384bcc54899ba04cc4995da11dd/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:29.084098 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f112f5-a26b-4e45-ba50-f41764dda933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7336552c874aa9a766288070767ae3, user_id=7975c384bcc54899ba04cc4995da11dd, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:29.087539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f112f5-a26b-4e45-ba50-f41764dda933 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:29.110837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1050/2113] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:28 2022] PUT /identity/v3/projects/4b7336552c874aa9a766288070767ae3/users/7975c384bcc54899ba04cc4995da11dd/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:29.124518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148c76c-0c18-46f5-b5fe-920cd0503008 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:29.124809 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148c76c-0c18-46f5-b5fe-920cd0503008 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:29.125328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148c76c-0c18-46f5-b5fe-920cd0503008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:29.132437 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1064/2114] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:29.145109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd62b2-9b3d-4cfc-afb4-bdda56dca773 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:29.146820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b1dd62b2-9b3d-4cfc-afb4-bdda56dca773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:29.167061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-7148c76c-0c18-46f5-b5fe-920cd0503008 None None] MFA Rules not processed for user `7975c384bcc54899ba04cc4995da11dd`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:29.168709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-7148c76c-0c18-46f5-b5fe-920cd0503008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:29.184733 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:29.263927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd62b2-9b3d-4cfc-afb4-bdda56dca773 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:29.286470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1051/2115] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:51:29.301543 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148c76c-0c18-46f5-b5fe-920cd0503008 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:29.303141 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-7148c76c-0c18-46f5-b5fe-920cd0503008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:29.390318 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148c76c-0c18-46f5-b5fe-920cd0503008 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:29.392217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277d2fa-e25a-470f-95f6-1063f393b179 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4799b0c9e5aa47d389121d2183666d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:29.393073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277d2fa-e25a-470f-95f6-1063f393b179 service glance] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:29.393489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277d2fa-e25a-470f-95f6-1063f393b179 service glance] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:29.393872 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277d2fa-e25a-470f-95f6-1063f393b179 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:29.395718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d27c2-1bf2-437c-b577-923f52b92092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:29.396440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d27c2-1bf2-437c-b577-923f52b92092 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:29.396877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d27c2-1bf2-437c-b577-923f52b92092 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:29.397214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d27c2-1bf2-437c-b577-923f52b92092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:29.414472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d27c2-1bf2-437c-b577-923f52b92092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:29.418133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d27c2-1bf2-437c-b577-923f52b92092 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:29.452837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1052/2116] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:29.484082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d27c2-1bf2-437c-b577-923f52b92092 service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:29.490971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:29.587741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277d2fa-e25a-470f-95f6-1063f393b179 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:29.606597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277d2fa-e25a-470f-95f6-1063f393b179 service glance] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:29.621057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1065/2117] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:51:29.683457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d27c2-1bf2-437c-b577-923f52b92092 service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:29.688884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3ee4d-1af0-4473-a3ad-be78c8579e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:29.689532 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3ee4d-1af0-4473-a3ad-be78c8579e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:29.689814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3ee4d-1af0-4473-a3ad-be78c8579e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:29.690086 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3ee4d-1af0-4473-a3ad-be78c8579e4d admin admin] PATH_INFO: `/v3/users/7975c384bcc54899ba04cc4995da11dd` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:29.691374 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3ee4d-1af0-4473-a3ad-be78c8579e4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7975c384bcc54899ba04cc4995da11dd)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:29.693419 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3ee4d-1af0-4473-a3ad-be78c8579e4d admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:29.761498 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1ef3ee4d-1af0-4473-a3ad-be78c8579e4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7975c384bcc54899ba04cc4995da11dd'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:29.771720 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1ef3ee4d-1af0-4473-a3ad-be78c8579e4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7975c384bcc54899ba04cc4995da11dd'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:29.790030 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1ef3ee4d-1af0-4473-a3ad-be78c8579e4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7975c384bcc54899ba04cc4995da11dd'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:29.802079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1ef3ee4d-1af0-4473-a3ad-be78c8579e4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7975c384bcc54899ba04cc4995da11dd'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:29.802980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1ef3ee4d-1af0-4473-a3ad-be78c8579e4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7975c384bcc54899ba04cc4995da11dd'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:29.812327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1053/2118] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:29 2022] DELETE /identity/v3/users/7975c384bcc54899ba04cc4995da11dd => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:29.814782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102b499-5e28-4ff7-b9a5-62082930baad None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:29.815320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102b499-5e28-4ff7-b9a5-62082930baad None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:29.815680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102b499-5e28-4ff7-b9a5-62082930baad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:29.843430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-0102b499-5e28-4ff7-b9a5-62082930baad None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:29.844919 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0102b499-5e28-4ff7-b9a5-62082930baad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:29.952157 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1066/2119] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:29.962171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3ee4d-1af0-4473-a3ad-be78c8579e4d admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:29.963791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1ef3ee4d-1af0-4473-a3ad-be78c8579e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:30.032497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3ee4d-1af0-4473-a3ad-be78c8579e4d admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:30.037651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcc6d1-7758-4d54-b242-1896af145c4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:30.038661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcc6d1-7758-4d54-b242-1896af145c4a service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:30.038952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcc6d1-7758-4d54-b242-1896af145c4a service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:30.039297 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcc6d1-7758-4d54-b242-1896af145c4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:30.053508 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bcc6d1-7758-4d54-b242-1896af145c4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:30.055178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bcc6d1-7758-4d54-b242-1896af145c4a service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:30.068485 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1054/2120] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:30.416495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102b499-5e28-4ff7-b9a5-62082930baad None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:30.418553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0102b499-5e28-4ff7-b9a5-62082930baad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:30.431374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102b499-5e28-4ff7-b9a5-62082930baad None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:30.446406 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a893ff-cd2b-4bdd-b580-aa7d2cb3c2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:30.447037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a893ff-cd2b-4bdd-b580-aa7d2cb3c2fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:30.447326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a893ff-cd2b-4bdd-b580-aa7d2cb3c2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:30.447607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a893ff-cd2b-4bdd-b580-aa7d2cb3c2fa admin admin] PATH_INFO: `/v3/projects/4b7336552c874aa9a766288070767ae3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:30.448959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a893ff-cd2b-4bdd-b580-aa7d2cb3c2fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b7336552c874aa9a766288070767ae3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:30.452121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a893ff-cd2b-4bdd-b580-aa7d2cb3c2fa admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:30.465051 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:30.502644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-81a893ff-cd2b-4bdd-b580-aa7d2cb3c2fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b7336552c874aa9a766288070767ae3', 'request_id': 'req-81a893ff-cd2b-4bdd-b580-aa7d2cb3c2fa'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:30.508963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-81a893ff-cd2b-4bdd-b580-aa7d2cb3c2fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b7336552c874aa9a766288070767ae3', 'request_id': 'req-81a893ff-cd2b-4bdd-b580-aa7d2cb3c2fa'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:30.517448 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-81a893ff-cd2b-4bdd-b580-aa7d2cb3c2fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b7336552c874aa9a766288070767ae3', 'request_id': 'req-81a893ff-cd2b-4bdd-b580-aa7d2cb3c2fa'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:30.531420 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-81a893ff-cd2b-4bdd-b580-aa7d2cb3c2fa admin admin] The token cache is being invalidate because project 4b7336552c874aa9a766288070767ae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:30.531708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-81a893ff-cd2b-4bdd-b580-aa7d2cb3c2fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:30.532421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-81a893ff-cd2b-4bdd-b580-aa7d2cb3c2fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:30.545319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1067/2121] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:30 2022] DELETE /identity/v3/projects/4b7336552c874aa9a766288070767ae3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:30.549274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b45c73-1808-49f4-8dcb-ae838178f9a2 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:30.550472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b45c73-1808-49f4-8dcb-ae838178f9a2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:30.550861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b45c73-1808-49f4-8dcb-ae838178f9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:30.630064 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-e3b45c73-1808-49f4-8dcb-ae838178f9a2 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:30.633530 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e3b45c73-1808-49f4-8dcb-ae838178f9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:30.690228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:30.784697 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1055/2122] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:30.791154 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee302d-7e1c-49be-a696-b2dba7189c18 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:30.791357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee302d-7e1c-49be-a696-b2dba7189c18 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:30.791478 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee302d-7e1c-49be-a696-b2dba7189c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:30.828719 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-07ee302d-7e1c-49be-a696-b2dba7189c18 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:30.829942 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-07ee302d-7e1c-49be-a696-b2dba7189c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:30.840984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1068/2123] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:51:30.848065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b45c73-1808-49f4-8dcb-ae838178f9a2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:30.862013 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b45c73-1808-49f4-8dcb-ae838178f9a2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:30.868638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64117b7f-cabb-450f-b741-25b1db480961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:30.869280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64117b7f-cabb-450f-b741-25b1db480961 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:30.869662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64117b7f-cabb-450f-b741-25b1db480961 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:30.869950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64117b7f-cabb-450f-b741-25b1db480961 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:30.870748 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64117b7f-cabb-450f-b741-25b1db480961 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:51:30.871068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64117b7f-cabb-450f-b741-25b1db480961 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:30.874223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64117b7f-cabb-450f-b741-25b1db480961 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:30.901275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1056/2124] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:51:30.905004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee302d-7e1c-49be-a696-b2dba7189c18 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:30.917328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee302d-7e1c-49be-a696-b2dba7189c18 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:30.934905 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8763f-4339-4396-83b7-4f16ca820456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:30.936677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8763f-4339-4396-83b7-4f16ca820456 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:30.937180 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8763f-4339-4396-83b7-4f16ca820456 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:30.937664 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8763f-4339-4396-83b7-4f16ca820456 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:30.938787 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8763f-4339-4396-83b7-4f16ca820456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:30.941791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8763f-4339-4396-83b7-4f16ca820456 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:31.003305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1069/2125] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:30 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:31.008900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64117b7f-cabb-450f-b741-25b1db480961 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:31.024903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64117b7f-cabb-450f-b741-25b1db480961 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:31.140017 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4602cf-60c8-473d-8bb1-cbca388a0b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:31.141126 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4602cf-60c8-473d-8bb1-cbca388a0b80 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:31.141636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4602cf-60c8-473d-8bb1-cbca388a0b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:31.142140 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4602cf-60c8-473d-8bb1-cbca388a0b80 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:31.143370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4602cf-60c8-473d-8bb1-cbca388a0b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:31.147084 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4602cf-60c8-473d-8bb1-cbca388a0b80 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:31.192126 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1057/2126] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:31 2022] POST /identity/v3/users => generated 288 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:31.196186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8763f-4339-4396-83b7-4f16ca820456 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:31.209103 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8763f-4339-4396-83b7-4f16ca820456 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:31.214047 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ada0e-a466-4601-9715-b5f9cda618c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:31.214753 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ada0e-a466-4601-9715-b5f9cda618c2 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:31.215954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ada0e-a466-4601-9715-b5f9cda618c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:31.220532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ada0e-a466-4601-9715-b5f9cda618c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:31.224840 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ada0e-a466-4601-9715-b5f9cda618c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:31.226695 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ada0e-a466-4601-9715-b5f9cda618c2 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:31.252853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1070/2127] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:31.255885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4602cf-60c8-473d-8bb1-cbca388a0b80 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:31.282862 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4602cf-60c8-473d-8bb1-cbca388a0b80 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:31.289738 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c508b-bdf2-4bb1-81da-20ea9b5cdb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:31.290411 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c508b-bdf2-4bb1-81da-20ea9b5cdb73 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:31.290744 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c508b-bdf2-4bb1-81da-20ea9b5cdb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:31.291855 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c508b-bdf2-4bb1-81da-20ea9b5cdb73 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:31.292621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c508b-bdf2-4bb1-81da-20ea9b5cdb73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:31.296479 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c508b-bdf2-4bb1-81da-20ea9b5cdb73 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:31.314937 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1058/2128] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:51:31.323317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ada0e-a466-4601-9715-b5f9cda618c2 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:31.336616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ada0e-a466-4601-9715-b5f9cda618c2 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:31.342677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d43608-47af-4492-a3b5-b62baaf91598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:31.343482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d43608-47af-4492-a3b5-b62baaf91598 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:31.343767 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d43608-47af-4492-a3b5-b62baaf91598 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:31.344192 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d43608-47af-4492-a3b5-b62baaf91598 admin admin] PATH_INFO: `/v3/projects/95194680c11a45f19195a60cdeae976c/users/590a1bd68bd945349ba141c372d9d662/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:31.376968 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d43608-47af-4492-a3b5-b62baaf91598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95194680c11a45f19195a60cdeae976c, user_id=590a1bd68bd945349ba141c372d9d662, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:31.380744 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d43608-47af-4492-a3b5-b62baaf91598 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:31.417395 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1071/2129] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:31 2022] PUT /identity/v3/projects/95194680c11a45f19195a60cdeae976c/users/590a1bd68bd945349ba141c372d9d662/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:31.418069 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37139ac7-50ba-4a40-9e08-94fd335eb99f None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:31.418211 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37139ac7-50ba-4a40-9e08-94fd335eb99f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:31.418517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37139ac7-50ba-4a40-9e08-94fd335eb99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:31.462140 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-37139ac7-50ba-4a40-9e08-94fd335eb99f None None] MFA Rules not processed for user `590a1bd68bd945349ba141c372d9d662`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:31.463407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-37139ac7-50ba-4a40-9e08-94fd335eb99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:31.601379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1059/2130] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:31.609860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d43608-47af-4492-a3b5-b62baaf91598 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:31.613815 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a1d43608-47af-4492-a3b5-b62baaf91598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:31.745324 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d43608-47af-4492-a3b5-b62baaf91598 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:31.753854 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3b7be-31ad-4bd2-a362-e5756f5341f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:31.757386 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3b7be-31ad-4bd2-a362-e5756f5341f9 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:31.757971 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3b7be-31ad-4bd2-a362-e5756f5341f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:31.758550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3b7be-31ad-4bd2-a362-e5756f5341f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:31.777029 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3b7be-31ad-4bd2-a362-e5756f5341f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:31.781693 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3b7be-31ad-4bd2-a362-e5756f5341f9 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:31.800989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1072/2131] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:32.085877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37139ac7-50ba-4a40-9e08-94fd335eb99f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:32.087768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-37139ac7-50ba-4a40-9e08-94fd335eb99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:32.108733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37139ac7-50ba-4a40-9e08-94fd335eb99f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:32.113672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66be0c2-dc12-48d8-9024-67dc0a06eb72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:32.114582 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be0c2-dc12-48d8-9024-67dc0a06eb72 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:32.114958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be0c2-dc12-48d8-9024-67dc0a06eb72 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:32.115243 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be0c2-dc12-48d8-9024-67dc0a06eb72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:32.119110 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d66be0c2-dc12-48d8-9024-67dc0a06eb72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:32.142965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:32.215652 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66be0c2-dc12-48d8-9024-67dc0a06eb72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:32.217540 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66be0c2-dc12-48d8-9024-67dc0a06eb72 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:32.242393 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1060/2132] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:33.149348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3b7be-31ad-4bd2-a362-e5756f5341f9 service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:33.158129 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3b7be-31ad-4bd2-a362-e5756f5341f9 service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:33.163168 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f1843-a55a-454f-a3a3-3f6ac36dd01e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:33.163822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f1843-a55a-454f-a3a3-3f6ac36dd01e service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:33.164108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f1843-a55a-454f-a3a3-3f6ac36dd01e service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:33.164427 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f1843-a55a-454f-a3a3-3f6ac36dd01e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:33.166228 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1b1f1843-a55a-454f-a3a3-3f6ac36dd01e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:33.213738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:33.239969 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f1843-a55a-454f-a3a3-3f6ac36dd01e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:33.241486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f1843-a55a-454f-a3a3-3f6ac36dd01e service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:33.251987 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1073/2133] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:42.223949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=19 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:43.246696 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=12 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:53.228702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66be0c2-dc12-48d8-9024-67dc0a06eb72 service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:53.230160 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d66be0c2-dc12-48d8-9024-67dc0a06eb72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:53.301000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66be0c2-dc12-48d8-9024-67dc0a06eb72 service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:53.306765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e222e4-d88a-4fc4-a40d-4e3373315569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:53.307457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e222e4-d88a-4fc4-a40d-4e3373315569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:53.307739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e222e4-d88a-4fc4-a40d-4e3373315569 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:53.308039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e222e4-d88a-4fc4-a40d-4e3373315569 admin admin] PATH_INFO: `/v3/users/5c66617ce8844ccbb40a5f906add4352` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:53.309399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e222e4-d88a-4fc4-a40d-4e3373315569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c66617ce8844ccbb40a5f906add4352)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:53.311846 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e222e4-d88a-4fc4-a40d-4e3373315569 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:53.375719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-12e222e4-d88a-4fc4-a40d-4e3373315569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c66617ce8844ccbb40a5f906add4352'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:53.383520 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-12e222e4-d88a-4fc4-a40d-4e3373315569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c66617ce8844ccbb40a5f906add4352'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:53.397713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-12e222e4-d88a-4fc4-a40d-4e3373315569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c66617ce8844ccbb40a5f906add4352'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:53.405914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-12e222e4-d88a-4fc4-a40d-4e3373315569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c66617ce8844ccbb40a5f906add4352'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:53.406612 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-12e222e4-d88a-4fc4-a40d-4e3373315569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c66617ce8844ccbb40a5f906add4352'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:53.412534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1061/2134] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:53 2022] DELETE /identity/v3/users/5c66617ce8844ccbb40a5f906add4352 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:53.425705 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c7ef0-7590-422c-8859-2b3e302b0690 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:53.426718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c7ef0-7590-422c-8859-2b3e302b0690 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:53.426718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c7ef0-7590-422c-8859-2b3e302b0690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:53.452761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-0a7c7ef0-7590-422c-8859-2b3e302b0690 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:53.453978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0a7c7ef0-7590-422c-8859-2b3e302b0690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:53.516447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1074/2135] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:53.531823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e222e4-d88a-4fc4-a40d-4e3373315569 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:53.533742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-12e222e4-d88a-4fc4-a40d-4e3373315569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:53.601856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e222e4-d88a-4fc4-a40d-4e3373315569 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:53.605621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c819ba-8c6a-4402-8c57-8f910284c9a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:53.606326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c819ba-8c6a-4402-8c57-8f910284c9a8 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:53.606678 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c819ba-8c6a-4402-8c57-8f910284c9a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:53.607038 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c819ba-8c6a-4402-8c57-8f910284c9a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:53.617665 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c819ba-8c6a-4402-8c57-8f910284c9a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:53.619223 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c819ba-8c6a-4402-8c57-8f910284c9a8 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:53.630295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1062/2136] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:53.939236 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c7ef0-7590-422c-8859-2b3e302b0690 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:53.941692 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0a7c7ef0-7590-422c-8859-2b3e302b0690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:53.955770 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c7ef0-7590-422c-8859-2b3e302b0690 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:53.960539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1213aa8-20e3-492e-a545-7bca4ff4ef25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:53.961183 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1213aa8-20e3-492e-a545-7bca4ff4ef25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:53.961466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1213aa8-20e3-492e-a545-7bca4ff4ef25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:53.961738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1213aa8-20e3-492e-a545-7bca4ff4ef25 admin admin] PATH_INFO: `/v3/projects/57b1d50a5fe247d4a52496f4079211e2` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:53.963426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1213aa8-20e3-492e-a545-7bca4ff4ef25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57b1d50a5fe247d4a52496f4079211e2)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:53.964935 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1213aa8-20e3-492e-a545-7bca4ff4ef25 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:54.014802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b1213aa8-20e3-492e-a545-7bca4ff4ef25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57b1d50a5fe247d4a52496f4079211e2', 'request_id': 'req-b1213aa8-20e3-492e-a545-7bca4ff4ef25'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.020478 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b1213aa8-20e3-492e-a545-7bca4ff4ef25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57b1d50a5fe247d4a52496f4079211e2', 'request_id': 'req-b1213aa8-20e3-492e-a545-7bca4ff4ef25'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.029832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b1213aa8-20e3-492e-a545-7bca4ff4ef25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57b1d50a5fe247d4a52496f4079211e2', 'request_id': 'req-b1213aa8-20e3-492e-a545-7bca4ff4ef25'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.041361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b1213aa8-20e3-492e-a545-7bca4ff4ef25 admin admin] The token cache is being invalidate because project 57b1d50a5fe247d4a52496f4079211e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:54.041937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b1213aa8-20e3-492e-a545-7bca4ff4ef25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.042646 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b1213aa8-20e3-492e-a545-7bca4ff4ef25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.044191 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1075/2137] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:53 2022] DELETE /identity/v3/projects/57b1d50a5fe247d4a52496f4079211e2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:54.048887 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c819ba-8c6a-4402-8c57-8f910284c9a8 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:54.059624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c819ba-8c6a-4402-8c57-8f910284c9a8 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:54.081288 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:54.144441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462f1ac-3135-4f2a-833a-c90fa746d0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:54.145232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462f1ac-3135-4f2a-833a-c90fa746d0a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:54.145568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462f1ac-3135-4f2a-833a-c90fa746d0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:54.145945 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462f1ac-3135-4f2a-833a-c90fa746d0a6 admin admin] PATH_INFO: `/v3/users/c5b28d90873d4f408ec0e5e70f17b583` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:54.147203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462f1ac-3135-4f2a-833a-c90fa746d0a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5b28d90873d4f408ec0e5e70f17b583)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:54.149295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462f1ac-3135-4f2a-833a-c90fa746d0a6 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:54.234837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-0462f1ac-3135-4f2a-833a-c90fa746d0a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5b28d90873d4f408ec0e5e70f17b583'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.243817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-0462f1ac-3135-4f2a-833a-c90fa746d0a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5b28d90873d4f408ec0e5e70f17b583'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.257778 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-0462f1ac-3135-4f2a-833a-c90fa746d0a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5b28d90873d4f408ec0e5e70f17b583'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.266420 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-0462f1ac-3135-4f2a-833a-c90fa746d0a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5b28d90873d4f408ec0e5e70f17b583'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.267151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-0462f1ac-3135-4f2a-833a-c90fa746d0a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5b28d90873d4f408ec0e5e70f17b583'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.275417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1063/2138] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:54 2022] DELETE /identity/v3/users/c5b28d90873d4f408ec0e5e70f17b583 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:51:54.583867 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1213aa8-20e3-492e-a545-7bca4ff4ef25 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:54.586707 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b1213aa8-20e3-492e-a545-7bca4ff4ef25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:54.608247 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1213aa8-20e3-492e-a545-7bca4ff4ef25 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:54.689923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36893749-b3ee-4f33-a947-f110653d72c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:54.690579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36893749-b3ee-4f33-a947-f110653d72c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:54.691010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36893749-b3ee-4f33-a947-f110653d72c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:54.691295 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36893749-b3ee-4f33-a947-f110653d72c9 admin admin] PATH_INFO: `/v3/projects/9688704c47a04b91affe95a40d1a8908` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:54.692536 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36893749-b3ee-4f33-a947-f110653d72c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9688704c47a04b91affe95a40d1a8908)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:54.694552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36893749-b3ee-4f33-a947-f110653d72c9 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:54.735134 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:54.756841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-36893749-b3ee-4f33-a947-f110653d72c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9688704c47a04b91affe95a40d1a8908', 'request_id': 'req-36893749-b3ee-4f33-a947-f110653d72c9'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.763397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-36893749-b3ee-4f33-a947-f110653d72c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9688704c47a04b91affe95a40d1a8908', 'request_id': 'req-36893749-b3ee-4f33-a947-f110653d72c9'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.772923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-36893749-b3ee-4f33-a947-f110653d72c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9688704c47a04b91affe95a40d1a8908', 'request_id': 'req-36893749-b3ee-4f33-a947-f110653d72c9'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.788463 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-36893749-b3ee-4f33-a947-f110653d72c9 admin admin] The token cache is being invalidate because project 9688704c47a04b91affe95a40d1a8908 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:54.788805 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-36893749-b3ee-4f33-a947-f110653d72c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.789561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-36893749-b3ee-4f33-a947-f110653d72c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.791554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1076/2139] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:54 2022] DELETE /identity/v3/projects/9688704c47a04b91affe95a40d1a8908 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:51:54.795788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462f1ac-3135-4f2a-833a-c90fa746d0a6 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:54.797550 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-0462f1ac-3135-4f2a-833a-c90fa746d0a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:54.812331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462f1ac-3135-4f2a-833a-c90fa746d0a6 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:54.896227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da60970-d690-4050-91d3-879324a73cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:54.896917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da60970-d690-4050-91d3-879324a73cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:54.897199 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da60970-d690-4050-91d3-879324a73cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:54.897475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da60970-d690-4050-91d3-879324a73cf0 admin admin] PATH_INFO: `/v3/users/d867d93d59ce4ed78ce24e99671367b0` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:54.898820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da60970-d690-4050-91d3-879324a73cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d867d93d59ce4ed78ce24e99671367b0)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:54.901435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da60970-d690-4050-91d3-879324a73cf0 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:54.966703 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:54.969891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-0da60970-d690-4050-91d3-879324a73cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd867d93d59ce4ed78ce24e99671367b0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:54.988770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-0da60970-d690-4050-91d3-879324a73cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd867d93d59ce4ed78ce24e99671367b0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:55.007112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-0da60970-d690-4050-91d3-879324a73cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd867d93d59ce4ed78ce24e99671367b0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:55.016973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-0da60970-d690-4050-91d3-879324a73cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd867d93d59ce4ed78ce24e99671367b0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:55.017732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-0da60970-d690-4050-91d3-879324a73cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd867d93d59ce4ed78ce24e99671367b0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:55.024082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1064/2140] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:51:54 2022] DELETE /identity/v3/users/d867d93d59ce4ed78ce24e99671367b0 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:51:55.384733 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36893749-b3ee-4f33-a947-f110653d72c9 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:55.386280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-36893749-b3ee-4f33-a947-f110653d72c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:55.409629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36893749-b3ee-4f33-a947-f110653d72c9 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:55.498996 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c471a-8e83-47cd-a085-48ee2bf0eda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:55.499659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c471a-8e83-47cd-a085-48ee2bf0eda5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:55.499948 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c471a-8e83-47cd-a085-48ee2bf0eda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:55.500236 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c471a-8e83-47cd-a085-48ee2bf0eda5 admin admin] PATH_INFO: `/v3/projects/84f9a168b3cb46569731ca18e08aa742` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:55.501471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c471a-8e83-47cd-a085-48ee2bf0eda5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f9a168b3cb46569731ca18e08aa742)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:55.503426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c471a-8e83-47cd-a085-48ee2bf0eda5 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:55.562620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8b9c471a-8e83-47cd-a085-48ee2bf0eda5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f9a168b3cb46569731ca18e08aa742', 'request_id': 'req-8b9c471a-8e83-47cd-a085-48ee2bf0eda5'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:55.568313 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8b9c471a-8e83-47cd-a085-48ee2bf0eda5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f9a168b3cb46569731ca18e08aa742', 'request_id': 'req-8b9c471a-8e83-47cd-a085-48ee2bf0eda5'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:55.576714 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8b9c471a-8e83-47cd-a085-48ee2bf0eda5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f9a168b3cb46569731ca18e08aa742', 'request_id': 'req-8b9c471a-8e83-47cd-a085-48ee2bf0eda5'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:55.590303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8b9c471a-8e83-47cd-a085-48ee2bf0eda5 admin admin] The token cache is being invalidate because project 84f9a168b3cb46569731ca18e08aa742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:51:55.590904 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8b9c471a-8e83-47cd-a085-48ee2bf0eda5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:55.591739 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8b9c471a-8e83-47cd-a085-48ee2bf0eda5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:51:55.593494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1077/2141] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:51:55 2022] DELETE /identity/v3/projects/84f9a168b3cb46569731ca18e08aa742 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:51:55.598432 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a7de4-8bda-4f18-acbf-2f0fa52fb316 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:55.598876 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a7de4-8bda-4f18-acbf-2f0fa52fb316 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:55.598993 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a7de4-8bda-4f18-acbf-2f0fa52fb316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:55.619125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ff8a7de4-8bda-4f18-acbf-2f0fa52fb316 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:55.620371 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ff8a7de4-8bda-4f18-acbf-2f0fa52fb316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:55.680526 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1065/2142] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:55.689809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb4bba-f5cf-4946-b9e3-0921cce87f56 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:55.690091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb4bba-f5cf-4946-b9e3-0921cce87f56 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:55.690391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb4bba-f5cf-4946-b9e3-0921cce87f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:55.712436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-51eb4bba-f5cf-4946-b9e3-0921cce87f56 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:55.713644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-51eb4bba-f5cf-4946-b9e3-0921cce87f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:55.724984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1078/2143] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:55.728876 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a7de4-8bda-4f18-acbf-2f0fa52fb316 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:55.740517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a7de4-8bda-4f18-acbf-2f0fa52fb316 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:55.747803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83622ed9-4c28-42e1-9392-32dedbfd7c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:55.748698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83622ed9-4c28-42e1-9392-32dedbfd7c0f admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:55.749080 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83622ed9-4c28-42e1-9392-32dedbfd7c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:55.749453 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83622ed9-4c28-42e1-9392-32dedbfd7c0f admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:55.750288 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83622ed9-4c28-42e1-9392-32dedbfd7c0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:51:55.750694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83622ed9-4c28-42e1-9392-32dedbfd7c0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:55.752466 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83622ed9-4c28-42e1-9392-32dedbfd7c0f admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:55.767152 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1066/2144] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:51:55.771518 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb4bba-f5cf-4946-b9e3-0921cce87f56 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:55.782199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb4bba-f5cf-4946-b9e3-0921cce87f56 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:55.789896 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c6145-7d04-46d0-9628-6e0f207a3ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:55.790599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c6145-7d04-46d0-9628-6e0f207a3ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:55.790958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c6145-7d04-46d0-9628-6e0f207a3ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:55.791294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c6145-7d04-46d0-9628-6e0f207a3ec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:55.791983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c6145-7d04-46d0-9628-6e0f207a3ec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:55.794470 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c6145-7d04-46d0-9628-6e0f207a3ec3 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:55.828453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1079/2145] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:51:55 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:51:55.831823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83622ed9-4c28-42e1-9392-32dedbfd7c0f admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:55.843798 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83622ed9-4c28-42e1-9392-32dedbfd7c0f admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:55.901105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89810d3d-1cd0-4b26-b6a6-e29e3997cd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:55.901727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89810d3d-1cd0-4b26-b6a6-e29e3997cd97 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:55.902007 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89810d3d-1cd0-4b26-b6a6-e29e3997cd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:55.902281 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89810d3d-1cd0-4b26-b6a6-e29e3997cd97 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:55.902969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89810d3d-1cd0-4b26-b6a6-e29e3997cd97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:55.905672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89810d3d-1cd0-4b26-b6a6-e29e3997cd97 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:55.933104 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1067/2146] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:51:55 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:51:55.934522 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c6145-7d04-46d0-9628-6e0f207a3ec3 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:55.946744 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c6145-7d04-46d0-9628-6e0f207a3ec3 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:55.952017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8794d-f929-42f6-9d13-1069cc69fcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:55.952663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8794d-f929-42f6-9d13-1069cc69fcf6 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:55.952947 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8794d-f929-42f6-9d13-1069cc69fcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:55.953222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8794d-f929-42f6-9d13-1069cc69fcf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:55.953936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8794d-f929-42f6-9d13-1069cc69fcf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:55.955559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8794d-f929-42f6-9d13-1069cc69fcf6 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:55.965920 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1080/2147] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:55.970839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89810d3d-1cd0-4b26-b6a6-e29e3997cd97 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:55.982644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89810d3d-1cd0-4b26-b6a6-e29e3997cd97 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:55.987406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf8d52-a165-4131-85fb-14a17a96d314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:55.988053 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf8d52-a165-4131-85fb-14a17a96d314 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:55.988370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf8d52-a165-4131-85fb-14a17a96d314 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:55.988651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf8d52-a165-4131-85fb-14a17a96d314 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:55.989481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf8d52-a165-4131-85fb-14a17a96d314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:55.991082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf8d52-a165-4131-85fb-14a17a96d314 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:56.005185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1068/2148] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:51:56.009351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8794d-f929-42f6-9d13-1069cc69fcf6 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:56.016721 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:56.021409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8794d-f929-42f6-9d13-1069cc69fcf6 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:56.028214 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cb934-332b-43ae-953c-8808e123dbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:56.029973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cb934-332b-43ae-953c-8808e123dbcf admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:56.030400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cb934-332b-43ae-953c-8808e123dbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:56.030748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cb934-332b-43ae-953c-8808e123dbcf admin admin] PATH_INFO: `/v3/projects/1fe2ae076eda4afa99c6c28464e1377b/users/9201a1d107bf41959751e7d7c4e4d37d/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:56.046869 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cb934-332b-43ae-953c-8808e123dbcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fe2ae076eda4afa99c6c28464e1377b, user_id=9201a1d107bf41959751e7d7c4e4d37d, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:56.049564 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cb934-332b-43ae-953c-8808e123dbcf admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:56.062963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1081/2149] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:51:56 2022] PUT /identity/v3/projects/1fe2ae076eda4afa99c6c28464e1377b/users/9201a1d107bf41959751e7d7c4e4d37d/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:51:56.068092 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b00753-d6c0-455f-8864-3255b5785825 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:56.068426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b00753-d6c0-455f-8864-3255b5785825 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:56.068709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b00753-d6c0-455f-8864-3255b5785825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:56.088395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-50b00753-d6c0-455f-8864-3255b5785825 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:56.089613 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-50b00753-d6c0-455f-8864-3255b5785825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:56.110472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:51:56.153100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1069/2150] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:56.164802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cb934-332b-43ae-953c-8808e123dbcf admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:56.166687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-171cb934-332b-43ae-953c-8808e123dbcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:56.230526 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cb934-332b-43ae-953c-8808e123dbcf admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:56.234794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8263d5-2bf0-4861-a92e-6d140c88f802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:56.235428 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8263d5-2bf0-4861-a92e-6d140c88f802 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:56.235712 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8263d5-2bf0-4861-a92e-6d140c88f802 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:56.235990 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8263d5-2bf0-4861-a92e-6d140c88f802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:56.245727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8263d5-2bf0-4861-a92e-6d140c88f802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:56.247070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8263d5-2bf0-4861-a92e-6d140c88f802 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:56.257674 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1082/2151] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:51:56.953415 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825e602-853a-416d-b9d2-4dca3534b42c None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:56.953967 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825e602-853a-416d-b9d2-4dca3534b42c None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:56.954204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825e602-853a-416d-b9d2-4dca3534b42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:56.987641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-a825e602-853a-416d-b9d2-4dca3534b42c None None] MFA Rules not processed for user `9201a1d107bf41959751e7d7c4e4d37d`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:51:56.989169 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a825e602-853a-416d-b9d2-4dca3534b42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:57.079977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1070/2152] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:51:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:51:57.090473 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8263d5-2bf0-4861-a92e-6d140c88f802 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:51:57.092031 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ee8263d5-2bf0-4861-a92e-6d140c88f802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:51:57.152694 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8263d5-2bf0-4861-a92e-6d140c88f802 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:51:57.156618 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3d897-a48e-4ee9-a2f8-96f41b6343ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:51:57.157190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3d897-a48e-4ee9-a2f8-96f41b6343ba service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:51:57.157473 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3d897-a48e-4ee9-a2f8-96f41b6343ba service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:51:57.157749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3d897-a48e-4ee9-a2f8-96f41b6343ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:51:57.166623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3d897-a48e-4ee9-a2f8-96f41b6343ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:51:57.168092 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3d897-a48e-4ee9-a2f8-96f41b6343ba service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:51:57.177305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1083/2153] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:51:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:52:05.011017 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb4c00-d40e-414e-acea-c0ccafe0b4cb None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:05.011532 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb4c00-d40e-414e-acea-c0ccafe0b4cb None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:05.011643 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb4c00-d40e-414e-acea-c0ccafe0b4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:05.031424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f8fb4c00-d40e-414e-acea-c0ccafe0b4cb None None] MFA Rules not processed for user `9201a1d107bf41959751e7d7c4e4d37d`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:05.032668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f8fb4c00-d40e-414e-acea-c0ccafe0b4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:05.039556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1071/2154] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:52:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:52:07.173829 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=60,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:52:08.031695 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3d897-a48e-4ee9-a2f8-96f41b6343ba service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:08.033476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d2b3d897-a48e-4ee9-a2f8-96f41b6343ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:08.053399 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3d897-a48e-4ee9-a2f8-96f41b6343ba service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:08.059733 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee6e99-ecfb-45f9-82c5-8895cb0d2501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:08.063378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee6e99-ecfb-45f9-82c5-8895cb0d2501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:08.063912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee6e99-ecfb-45f9-82c5-8895cb0d2501 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:08.064905 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee6e99-ecfb-45f9-82c5-8895cb0d2501 admin admin] PATH_INFO: `/v3/users/590a1bd68bd945349ba141c372d9d662` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:08.070976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee6e99-ecfb-45f9-82c5-8895cb0d2501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=590a1bd68bd945349ba141c372d9d662)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:08.074814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee6e99-ecfb-45f9-82c5-8895cb0d2501 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:08.184265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-82ee6e99-ecfb-45f9-82c5-8895cb0d2501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '590a1bd68bd945349ba141c372d9d662'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:08.200038 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-82ee6e99-ecfb-45f9-82c5-8895cb0d2501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '590a1bd68bd945349ba141c372d9d662'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:08.234987 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-82ee6e99-ecfb-45f9-82c5-8895cb0d2501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '590a1bd68bd945349ba141c372d9d662'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:08.247229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-82ee6e99-ecfb-45f9-82c5-8895cb0d2501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '590a1bd68bd945349ba141c372d9d662'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:08.249114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-82ee6e99-ecfb-45f9-82c5-8895cb0d2501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '590a1bd68bd945349ba141c372d9d662'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:08.266429 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1084/2155] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:52:08 2022] DELETE /identity/v3/users/590a1bd68bd945349ba141c372d9d662 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:52:08.271759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8793d-fc9f-4708-b801-9517580909fb None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:08.272240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8793d-fc9f-4708-b801-9517580909fb None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:08.273309 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8793d-fc9f-4708-b801-9517580909fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:08.328192 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-86c8793d-fc9f-4708-b801-9517580909fb None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:08.329573 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-86c8793d-fc9f-4708-b801-9517580909fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:08.460400 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1072/2156] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:52:08.489515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee6e99-ecfb-45f9-82c5-8895cb0d2501 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:08.491085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-82ee6e99-ecfb-45f9-82c5-8895cb0d2501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:08.595129 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee6e99-ecfb-45f9-82c5-8895cb0d2501 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:08.601641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32dcb9-4f0a-42d5-957f-557b65840bba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:08.602487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32dcb9-4f0a-42d5-957f-557b65840bba service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:08.603057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32dcb9-4f0a-42d5-957f-557b65840bba service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:08.603626 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32dcb9-4f0a-42d5-957f-557b65840bba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:08.616696 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32dcb9-4f0a-42d5-957f-557b65840bba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:08.618184 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32dcb9-4f0a-42d5-957f-557b65840bba service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:08.633969 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1085/2157] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:52:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:52:09.348019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8793d-fc9f-4708-b801-9517580909fb None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:09.349537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-86c8793d-fc9f-4708-b801-9517580909fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:09.363774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8793d-fc9f-4708-b801-9517580909fb None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:09.376589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d8a46-84dd-4826-ba85-77874321ba78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:09.377727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d8a46-84dd-4826-ba85-77874321ba78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:09.378164 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d8a46-84dd-4826-ba85-77874321ba78 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:09.378610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d8a46-84dd-4826-ba85-77874321ba78 admin admin] PATH_INFO: `/v3/projects/95194680c11a45f19195a60cdeae976c` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:09.383591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d8a46-84dd-4826-ba85-77874321ba78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95194680c11a45f19195a60cdeae976c)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:09.386901 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d8a46-84dd-4826-ba85-77874321ba78 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:09.448826 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:52:09.453099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c04d8a46-84dd-4826-ba85-77874321ba78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95194680c11a45f19195a60cdeae976c', 'request_id': 'req-c04d8a46-84dd-4826-ba85-77874321ba78'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:09.461170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c04d8a46-84dd-4826-ba85-77874321ba78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95194680c11a45f19195a60cdeae976c', 'request_id': 'req-c04d8a46-84dd-4826-ba85-77874321ba78'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:09.469419 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c04d8a46-84dd-4826-ba85-77874321ba78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95194680c11a45f19195a60cdeae976c', 'request_id': 'req-c04d8a46-84dd-4826-ba85-77874321ba78'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:09.489038 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c04d8a46-84dd-4826-ba85-77874321ba78 admin admin] The token cache is being invalidate because project 95194680c11a45f19195a60cdeae976c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:52:09.489819 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c04d8a46-84dd-4826-ba85-77874321ba78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:09.490771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c04d8a46-84dd-4826-ba85-77874321ba78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:09.507510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1073/2158] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:52:09 2022] DELETE /identity/v3/projects/95194680c11a45f19195a60cdeae976c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:52:10.503715 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32dcb9-4f0a-42d5-957f-557b65840bba service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:10.505470 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-3d32dcb9-4f0a-42d5-957f-557b65840bba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:10.519599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32dcb9-4f0a-42d5-957f-557b65840bba service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:10.593397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295da6fb-f7f3-43e4-ac61-917f0447376f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:10.594789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295da6fb-f7f3-43e4-ac61-917f0447376f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:10.595353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295da6fb-f7f3-43e4-ac61-917f0447376f admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:10.596082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295da6fb-f7f3-43e4-ac61-917f0447376f admin admin] PATH_INFO: `/v3/users/1f99fcc5eddd45c89abca70f00ad6fed` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:10.597941 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295da6fb-f7f3-43e4-ac61-917f0447376f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f99fcc5eddd45c89abca70f00ad6fed)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:10.600107 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295da6fb-f7f3-43e4-ac61-917f0447376f admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:10.623298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:52:10.673341 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-295da6fb-f7f3-43e4-ac61-917f0447376f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f99fcc5eddd45c89abca70f00ad6fed'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:10.691205 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-295da6fb-f7f3-43e4-ac61-917f0447376f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f99fcc5eddd45c89abca70f00ad6fed'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:10.706299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-295da6fb-f7f3-43e4-ac61-917f0447376f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f99fcc5eddd45c89abca70f00ad6fed'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:10.713566 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-295da6fb-f7f3-43e4-ac61-917f0447376f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f99fcc5eddd45c89abca70f00ad6fed'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:10.714298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-295da6fb-f7f3-43e4-ac61-917f0447376f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f99fcc5eddd45c89abca70f00ad6fed'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:10.722377 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1086/2159] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:52:10 2022] DELETE /identity/v3/users/1f99fcc5eddd45c89abca70f00ad6fed => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:52:10.905844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d8a46-84dd-4826-ba85-77874321ba78 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:10.907321 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c04d8a46-84dd-4826-ba85-77874321ba78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:10.936612 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d8a46-84dd-4826-ba85-77874321ba78 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:11.007003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea6ceb-5253-466b-9e02-e9996fbf0506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:11.007653 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea6ceb-5253-466b-9e02-e9996fbf0506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:11.007935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea6ceb-5253-466b-9e02-e9996fbf0506 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:11.008213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea6ceb-5253-466b-9e02-e9996fbf0506 admin admin] PATH_INFO: `/v3/projects/a9ad699f792e4c50a898079291f09254` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:11.009395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea6ceb-5253-466b-9e02-e9996fbf0506 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9ad699f792e4c50a898079291f09254)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:11.011289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea6ceb-5253-466b-9e02-e9996fbf0506 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:11.058729 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9aea6ceb-5253-466b-9e02-e9996fbf0506 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9ad699f792e4c50a898079291f09254', 'request_id': 'req-9aea6ceb-5253-466b-9e02-e9996fbf0506'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:11.064815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9aea6ceb-5253-466b-9e02-e9996fbf0506 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9ad699f792e4c50a898079291f09254', 'request_id': 'req-9aea6ceb-5253-466b-9e02-e9996fbf0506'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:11.070911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9aea6ceb-5253-466b-9e02-e9996fbf0506 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9ad699f792e4c50a898079291f09254', 'request_id': 'req-9aea6ceb-5253-466b-9e02-e9996fbf0506'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:11.085009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9aea6ceb-5253-466b-9e02-e9996fbf0506 admin admin] The token cache is being invalidate because project a9ad699f792e4c50a898079291f09254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:52:11.085302 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9aea6ceb-5253-466b-9e02-e9996fbf0506 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:11.085944 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9aea6ceb-5253-466b-9e02-e9996fbf0506 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:11.087464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1074/2160] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:52:10 2022] DELETE /identity/v3/projects/a9ad699f792e4c50a898079291f09254 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:52:11.092938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103cc58-8275-43ce-80dd-a077afa0a0d6 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:11.093233 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103cc58-8275-43ce-80dd-a077afa0a0d6 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:11.093519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103cc58-8275-43ce-80dd-a077afa0a0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:11.114396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-0103cc58-8275-43ce-80dd-a077afa0a0d6 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:11.115583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0103cc58-8275-43ce-80dd-a077afa0a0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:11.174954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1087/2161] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:52:11.182992 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7919be4-7703-4f72-a94c-4e9423d4e41b None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:11.183275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7919be4-7703-4f72-a94c-4e9423d4e41b None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:11.183554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7919be4-7703-4f72-a94c-4e9423d4e41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:11.206711 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-d7919be4-7703-4f72-a94c-4e9423d4e41b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:11.207932 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d7919be4-7703-4f72-a94c-4e9423d4e41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:11.215873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1075/2162] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:52:11.220161 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103cc58-8275-43ce-80dd-a077afa0a0d6 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:11.230573 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103cc58-8275-43ce-80dd-a077afa0a0d6 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:11.235738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99856197-67bd-4c8e-b95a-698c28caa6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:11.236369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99856197-67bd-4c8e-b95a-698c28caa6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:11.236652 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99856197-67bd-4c8e-b95a-698c28caa6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:11.236927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99856197-67bd-4c8e-b95a-698c28caa6e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:11.237620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99856197-67bd-4c8e-b95a-698c28caa6e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:52:11.237923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99856197-67bd-4c8e-b95a-698c28caa6e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:11.239517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99856197-67bd-4c8e-b95a-698c28caa6e7 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:11.252259 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1088/2163] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:52:11.259595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7919be4-7703-4f72-a94c-4e9423d4e41b None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:11.272254 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7919be4-7703-4f72-a94c-4e9423d4e41b None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:11.277276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d952b7-c59b-4c36-957e-9650ce68d373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:11.277906 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d952b7-c59b-4c36-957e-9650ce68d373 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:11.278192 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d952b7-c59b-4c36-957e-9650ce68d373 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:11.278498 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d952b7-c59b-4c36-957e-9650ce68d373 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:11.279307 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d952b7-c59b-4c36-957e-9650ce68d373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:11.281949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d952b7-c59b-4c36-957e-9650ce68d373 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:11.315312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1076/2164] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:52:11 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:52:11.316663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99856197-67bd-4c8e-b95a-698c28caa6e7 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:11.327987 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99856197-67bd-4c8e-b95a-698c28caa6e7 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:11.412235 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43df48-bc8a-4d90-999c-ae8d9f9975a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:11.412887 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43df48-bc8a-4d90-999c-ae8d9f9975a7 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:11.413168 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43df48-bc8a-4d90-999c-ae8d9f9975a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:11.413440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43df48-bc8a-4d90-999c-ae8d9f9975a7 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:11.414102 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43df48-bc8a-4d90-999c-ae8d9f9975a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:11.416010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43df48-bc8a-4d90-999c-ae8d9f9975a7 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:11.441691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1089/2165] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:52:11 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:52:11.445713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d952b7-c59b-4c36-957e-9650ce68d373 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:11.463258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d952b7-c59b-4c36-957e-9650ce68d373 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:11.475455 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984648fb-78b1-4f0b-9c1a-68c88905cca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:11.476203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984648fb-78b1-4f0b-9c1a-68c88905cca0 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:11.476518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984648fb-78b1-4f0b-9c1a-68c88905cca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:11.476859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984648fb-78b1-4f0b-9c1a-68c88905cca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:11.477533 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984648fb-78b1-4f0b-9c1a-68c88905cca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:11.479445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984648fb-78b1-4f0b-9c1a-68c88905cca0 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:11.494515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1077/2166] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:52:11.495383 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43df48-bc8a-4d90-999c-ae8d9f9975a7 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:11.509405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43df48-bc8a-4d90-999c-ae8d9f9975a7 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:11.515030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc0971-49e0-406a-a36f-23b41880fcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:11.515849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc0971-49e0-406a-a36f-23b41880fcda admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:11.516369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc0971-49e0-406a-a36f-23b41880fcda admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:11.516719 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc0971-49e0-406a-a36f-23b41880fcda admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:11.517521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc0971-49e0-406a-a36f-23b41880fcda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:11.519186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc0971-49e0-406a-a36f-23b41880fcda admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:11.525359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:52:11.536586 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1090/2167] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:52:11.539055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984648fb-78b1-4f0b-9c1a-68c88905cca0 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:11.554849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984648fb-78b1-4f0b-9c1a-68c88905cca0 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:11.561345 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c24df-83fc-495a-b66d-dbffe41fe80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:11.562031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c24df-83fc-495a-b66d-dbffe41fe80a admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:11.562322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c24df-83fc-495a-b66d-dbffe41fe80a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:11.562599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c24df-83fc-495a-b66d-dbffe41fe80a admin admin] PATH_INFO: `/v3/projects/5b71821ba42d4b9987a6fc6943c4f817/users/f366266902094cd49554b5b0459272fc/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:11.578821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c24df-83fc-495a-b66d-dbffe41fe80a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b71821ba42d4b9987a6fc6943c4f817, user_id=f366266902094cd49554b5b0459272fc, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:11.582046 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c24df-83fc-495a-b66d-dbffe41fe80a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:11.593957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:52:11.599648 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1078/2168] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:52:11 2022] PUT /identity/v3/projects/5b71821ba42d4b9987a6fc6943c4f817/users/f366266902094cd49554b5b0459272fc/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:52:11.610748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823c2e5-042f-4cf2-a51a-a8f79ec7228d None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:11.611097 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823c2e5-042f-4cf2-a51a-a8f79ec7228d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:11.611438 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823c2e5-042f-4cf2-a51a-a8f79ec7228d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:11.641534 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-3823c2e5-042f-4cf2-a51a-a8f79ec7228d None None] MFA Rules not processed for user `f366266902094cd49554b5b0459272fc`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:11.643697 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-3823c2e5-042f-4cf2-a51a-a8f79ec7228d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:11.724786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1091/2169] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:52:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:52:11.729623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c24df-83fc-495a-b66d-dbffe41fe80a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:11.803634 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c24df-83fc-495a-b66d-dbffe41fe80a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:11.809007 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53664e66-3dd0-46a4-914a-195d5e7b5e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:11.809683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53664e66-3dd0-46a4-914a-195d5e7b5e24 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:11.809995 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53664e66-3dd0-46a4-914a-195d5e7b5e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:11.810247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53664e66-3dd0-46a4-914a-195d5e7b5e24 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:11.811059 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53664e66-3dd0-46a4-914a-195d5e7b5e24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:11.813053 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53664e66-3dd0-46a4-914a-195d5e7b5e24 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:11.845610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1079/2170] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:52:11 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:52:11.847058 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823c2e5-042f-4cf2-a51a-a8f79ec7228d None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:11.858388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823c2e5-042f-4cf2-a51a-a8f79ec7228d None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:11.922453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08facef7-e69b-449f-aa23-a29ff2d91f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:11.923080 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08facef7-e69b-449f-aa23-a29ff2d91f6b admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:11.923363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08facef7-e69b-449f-aa23-a29ff2d91f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:11.923633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08facef7-e69b-449f-aa23-a29ff2d91f6b admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:11.924342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08facef7-e69b-449f-aa23-a29ff2d91f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:11.926487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08facef7-e69b-449f-aa23-a29ff2d91f6b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:11.948882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1092/2171] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:52:11 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:52:11.953246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53664e66-3dd0-46a4-914a-195d5e7b5e24 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:11.963959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53664e66-3dd0-46a4-914a-195d5e7b5e24 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:11.969669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89af01f-23bd-474d-94e5-8e86a395def3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:11.970299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89af01f-23bd-474d-94e5-8e86a395def3 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:11.970588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89af01f-23bd-474d-94e5-8e86a395def3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:11.970868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89af01f-23bd-474d-94e5-8e86a395def3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:11.971575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89af01f-23bd-474d-94e5-8e86a395def3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:11.973212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89af01f-23bd-474d-94e5-8e86a395def3 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:11.992536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1080/2172] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:52:11.996777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08facef7-e69b-449f-aa23-a29ff2d91f6b admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:12.007342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08facef7-e69b-449f-aa23-a29ff2d91f6b admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:12.011901 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6336e07-6573-452f-8884-234c3a38a9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:12.012594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6336e07-6573-452f-8884-234c3a38a9f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:12.012875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6336e07-6573-452f-8884-234c3a38a9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:12.013354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6336e07-6573-452f-8884-234c3a38a9f9 admin admin] PATH_INFO: `/v3/projects/d7204dc12c324849afab1defe67f0fd7/users/226fde94e62c4d9394846179aa2d0cd6/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:12.030161 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6336e07-6573-452f-8884-234c3a38a9f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7204dc12c324849afab1defe67f0fd7, user_id=226fde94e62c4d9394846179aa2d0cd6, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:12.032967 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6336e07-6573-452f-8884-234c3a38a9f9 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:12.047278 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1093/2173] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:52:11 2022] PUT /identity/v3/projects/d7204dc12c324849afab1defe67f0fd7/users/226fde94e62c4d9394846179aa2d0cd6/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:52:12.051231 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89af01f-23bd-474d-94e5-8e86a395def3 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:12.062108 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89af01f-23bd-474d-94e5-8e86a395def3 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:12.121983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fb06a-29f0-4a3f-81bd-8187a054c505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:12.122843 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fb06a-29f0-4a3f-81bd-8187a054c505 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:12.123135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fb06a-29f0-4a3f-81bd-8187a054c505 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:12.123444 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fb06a-29f0-4a3f-81bd-8187a054c505 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:12.124124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5fb06a-29f0-4a3f-81bd-8187a054c505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:12.125783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5fb06a-29f0-4a3f-81bd-8187a054c505 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:12.135971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1081/2174] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:52:12.140670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6336e07-6573-452f-8884-234c3a38a9f9 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:12.150865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6336e07-6573-452f-8884-234c3a38a9f9 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:12.156328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb0aaf-a1c8-4033-bd85-0dc675ef3f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:12.156992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb0aaf-a1c8-4033-bd85-0dc675ef3f46 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:12.157277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb0aaf-a1c8-4033-bd85-0dc675ef3f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:12.157556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb0aaf-a1c8-4033-bd85-0dc675ef3f46 admin admin] PATH_INFO: `/v3/projects/d7204dc12c324849afab1defe67f0fd7/users/226fde94e62c4d9394846179aa2d0cd6/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:12.159545 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb0aaf-a1c8-4033-bd85-0dc675ef3f46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7204dc12c324849afab1defe67f0fd7, user_id=226fde94e62c4d9394846179aa2d0cd6, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:12.162216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb0aaf-a1c8-4033-bd85-0dc675ef3f46 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:12.172826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1094/2175] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:52:12 2022] PUT /identity/v3/projects/d7204dc12c324849afab1defe67f0fd7/users/226fde94e62c4d9394846179aa2d0cd6/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:52:12.184269 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c917e-0bbd-46d5-ab32-86518f340832 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:12.184521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c917e-0bbd-46d5-ab32-86518f340832 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:12.184803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c917e-0bbd-46d5-ab32-86518f340832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:12.214868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-1a1c917e-0bbd-46d5-ab32-86518f340832 None None] MFA Rules not processed for user `226fde94e62c4d9394846179aa2d0cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:12.216100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1a1c917e-0bbd-46d5-ab32-86518f340832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:12.278101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:52:12.288322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1082/2176] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:52:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:52:12.299896 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb0aaf-a1c8-4033-bd85-0dc675ef3f46 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:12.301694 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-21bb0aaf-a1c8-4033-bd85-0dc675ef3f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:12.349174 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:52:12.361917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb0aaf-a1c8-4033-bd85-0dc675ef3f46 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:12.365751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349414f-2df7-43d9-b1ee-b1bf1ab2e1c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:12.366373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349414f-2df7-43d9-b1ee-b1bf1ab2e1c6 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:12.366657 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349414f-2df7-43d9-b1ee-b1bf1ab2e1c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:12.366935 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349414f-2df7-43d9-b1ee-b1bf1ab2e1c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:12.376553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349414f-2df7-43d9-b1ee-b1bf1ab2e1c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:12.378042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349414f-2df7-43d9-b1ee-b1bf1ab2e1c6 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:12.387913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1095/2177] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:52:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:52:12.467594 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c917e-0bbd-46d5-ab32-86518f340832 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:12.478165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c917e-0bbd-46d5-ab32-86518f340832 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:12.481896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6dcc2-7ff1-406b-bc36-37ba1e479a14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:12.482706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6dcc2-7ff1-406b-bc36-37ba1e479a14 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:12.483098 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6dcc2-7ff1-406b-bc36-37ba1e479a14 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:12.483440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6dcc2-7ff1-406b-bc36-37ba1e479a14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:12.550384 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6dcc2-7ff1-406b-bc36-37ba1e479a14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:12.551844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6dcc2-7ff1-406b-bc36-37ba1e479a14 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:12.564182 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1083/2178] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:52:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:52:18.590835 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34993a-7d38-4379-89ee-f7b49b32bbab None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:18.591294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34993a-7d38-4379-89ee-f7b49b32bbab None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:18.591380 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34993a-7d38-4379-89ee-f7b49b32bbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:18.613686 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-da34993a-7d38-4379-89ee-f7b49b32bbab None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:18.614900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-da34993a-7d38-4379-89ee-f7b49b32bbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:18.674524 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1096/2179] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Aug 3 18:52:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:52:18.717589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6dcc2-7ff1-406b-bc36-37ba1e479a14 service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:18.719158 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d6b6dcc2-7ff1-406b-bc36-37ba1e479a14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:18.789248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6dcc2-7ff1-406b-bc36-37ba1e479a14 service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:18.794327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bfbfb-56e3-43b7-ba8e-62e93957c30b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:18.795598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bfbfb-56e3-43b7-ba8e-62e93957c30b service swift] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:18.795877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bfbfb-56e3-43b7-ba8e-62e93957c30b service swift] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:18.796151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bfbfb-56e3-43b7-ba8e-62e93957c30b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:18.804300 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bfbfb-56e3-43b7-ba8e-62e93957c30b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:18.805891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bfbfb-56e3-43b7-ba8e-62e93957c30b service swift] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:18.813921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1084/2180] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 18:52:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 18:52:19.383741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b51c63-e4a8-439c-9318-4957c0a5dfb0 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:19.384782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b51c63-e4a8-439c-9318-4957c0a5dfb0 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:19.384782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b51c63-e4a8-439c-9318-4957c0a5dfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:19.405727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-10b51c63-e4a8-439c-9318-4957c0a5dfb0 None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:19.407087 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-10b51c63-e4a8-439c-9318-4957c0a5dfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:19.414533 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1097/2181] 10.222.0.69 () {58 vars in 967 bytes} [Wed Aug 3 18:52:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:52:19.427774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bfbfb-56e3-43b7-ba8e-62e93957c30b service swift] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:19.439777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bfbfb-56e3-43b7-ba8e-62e93957c30b service swift] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:19.445220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d131c-8125-4ab6-9714-41ea35c0f9fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:19.445835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d131c-8125-4ab6-9714-41ea35c0f9fd service swift] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:19.446115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d131c-8125-4ab6-9714-41ea35c0f9fd service swift] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:19.446389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d131c-8125-4ab6-9714-41ea35c0f9fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:19.456571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d131c-8125-4ab6-9714-41ea35c0f9fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:19.458153 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d131c-8125-4ab6-9714-41ea35c0f9fd service swift] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:19.466287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1085/2182] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 18:52:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 18:52:19.739025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe049c7d-ac7f-4ad2-9d8e-fc0414928b56 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:19.739025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe049c7d-ac7f-4ad2-9d8e-fc0414928b56 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:19.739025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe049c7d-ac7f-4ad2-9d8e-fc0414928b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:19.758990 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-fe049c7d-ac7f-4ad2-9d8e-fc0414928b56 None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:19.760178 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-fe049c7d-ac7f-4ad2-9d8e-fc0414928b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:19.767667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1098/2183] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Aug 3 18:52:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:52:19.779588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d131c-8125-4ab6-9714-41ea35c0f9fd service swift] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:19.790202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d131c-8125-4ab6-9714-41ea35c0f9fd service swift] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:19.796340 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6a37f-fe50-49f5-8baa-304ce699178a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:19.796956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6a37f-fe50-49f5-8baa-304ce699178a service swift] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:19.797236 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6a37f-fe50-49f5-8baa-304ce699178a service swift] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:19.797513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6a37f-fe50-49f5-8baa-304ce699178a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:19.805619 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6a37f-fe50-49f5-8baa-304ce699178a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:19.807190 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6a37f-fe50-49f5-8baa-304ce699178a service swift] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:19.816028 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1086/2184] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 18:52:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 18:52:20.158914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b1005-2c9e-46fc-8e71-e7b23df46372 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:20.159200 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b1005-2c9e-46fc-8e71-e7b23df46372 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:20.159474 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b1005-2c9e-46fc-8e71-e7b23df46372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:20.178780 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-8d5b1005-2c9e-46fc-8e71-e7b23df46372 None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:20.179971 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8d5b1005-2c9e-46fc-8e71-e7b23df46372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:20.187246 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1099/2185] 10.222.0.69 () {58 vars in 967 bytes} [Wed Aug 3 18:52:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:52:20.199206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6a37f-fe50-49f5-8baa-304ce699178a service swift] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:20.209860 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6a37f-fe50-49f5-8baa-304ce699178a service swift] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:20.214791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64fdcd-b1dd-421e-9764-2bb0a1688cd4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:20.215332 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64fdcd-b1dd-421e-9764-2bb0a1688cd4 service swift] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:20.215611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64fdcd-b1dd-421e-9764-2bb0a1688cd4 service swift] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:20.215883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64fdcd-b1dd-421e-9764-2bb0a1688cd4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:20.224720 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64fdcd-b1dd-421e-9764-2bb0a1688cd4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:20.226280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64fdcd-b1dd-421e-9764-2bb0a1688cd4 service swift] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:20.237325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1087/2186] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 18:52:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 18:52:23.413823 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b1005-2c9e-46fc-8e71-e7b23df46372 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:23.415450 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8d5b1005-2c9e-46fc-8e71-e7b23df46372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:23.496282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b1005-2c9e-46fc-8e71-e7b23df46372 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:23.501812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f9f55-b5c7-4a4e-8df1-abb229b0f6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:23.502460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f9f55-b5c7-4a4e-8df1-abb229b0f6ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:23.502741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f9f55-b5c7-4a4e-8df1-abb229b0f6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:23.503009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f9f55-b5c7-4a4e-8df1-abb229b0f6ed admin admin] PATH_INFO: `/v3/users/f09070fbc0dc4486ae03bae2701fe85c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:23.504476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f9f55-b5c7-4a4e-8df1-abb229b0f6ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=f09070fbc0dc4486ae03bae2701fe85c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:23.506398 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f9f55-b5c7-4a4e-8df1-abb229b0f6ed admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:23.559428 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-1d5f9f55-b5c7-4a4e-8df1-abb229b0f6ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f09070fbc0dc4486ae03bae2701fe85c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:23.566782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-1d5f9f55-b5c7-4a4e-8df1-abb229b0f6ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f09070fbc0dc4486ae03bae2701fe85c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:23.579830 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-1d5f9f55-b5c7-4a4e-8df1-abb229b0f6ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f09070fbc0dc4486ae03bae2701fe85c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:23.581039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:52:23.589121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-1d5f9f55-b5c7-4a4e-8df1-abb229b0f6ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f09070fbc0dc4486ae03bae2701fe85c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:23.590161 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-1d5f9f55-b5c7-4a4e-8df1-abb229b0f6ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f09070fbc0dc4486ae03bae2701fe85c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:23.597396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1100/2187] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:52:23 2022] DELETE /identity/v3/users/f09070fbc0dc4486ae03bae2701fe85c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:52:23.601872 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64575c35-953b-4d42-8494-c7c421e5c2b7 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:23.602326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64575c35-953b-4d42-8494-c7c421e5c2b7 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:23.602435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64575c35-953b-4d42-8494-c7c421e5c2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:23.622225 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-64575c35-953b-4d42-8494-c7c421e5c2b7 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:23.623465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-64575c35-953b-4d42-8494-c7c421e5c2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:23.666075 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:52:23.685701 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1088/2188] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:52:23.695570 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f9f55-b5c7-4a4e-8df1-abb229b0f6ed admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:23.697009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1d5f9f55-b5c7-4a4e-8df1-abb229b0f6ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:23.755451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f9f55-b5c7-4a4e-8df1-abb229b0f6ed admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:23.759091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e856bf-47fd-40c0-87eb-4e6f52480c37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:23.759706 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e856bf-47fd-40c0-87eb-4e6f52480c37 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:23.759987 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e856bf-47fd-40c0-87eb-4e6f52480c37 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:23.760261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e856bf-47fd-40c0-87eb-4e6f52480c37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:23.769393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e856bf-47fd-40c0-87eb-4e6f52480c37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:23.770823 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e856bf-47fd-40c0-87eb-4e6f52480c37 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:23.780701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1101/2189] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:52:23.973915 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64575c35-953b-4d42-8494-c7c421e5c2b7 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:23.975985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-64575c35-953b-4d42-8494-c7c421e5c2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:23.995928 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64575c35-953b-4d42-8494-c7c421e5c2b7 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:24.002750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1dc6fa-fb59-4949-ac5b-4e78c7948df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:24.003607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dc6fa-fb59-4949-ac5b-4e78c7948df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:24.004335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dc6fa-fb59-4949-ac5b-4e78c7948df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:24.004960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dc6fa-fb59-4949-ac5b-4e78c7948df7 admin admin] PATH_INFO: `/v3/projects/f15b6902ce664c0ba8fc4cb0868fdaa3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:24.007113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dc6fa-fb59-4949-ac5b-4e78c7948df7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f15b6902ce664c0ba8fc4cb0868fdaa3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:24.009129 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dc6fa-fb59-4949-ac5b-4e78c7948df7 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:24.062352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b1dc6fa-fb59-4949-ac5b-4e78c7948df7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f15b6902ce664c0ba8fc4cb0868fdaa3', 'request_id': 'req-1b1dc6fa-fb59-4949-ac5b-4e78c7948df7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:24.068772 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b1dc6fa-fb59-4949-ac5b-4e78c7948df7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f15b6902ce664c0ba8fc4cb0868fdaa3', 'request_id': 'req-1b1dc6fa-fb59-4949-ac5b-4e78c7948df7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:24.075446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b1dc6fa-fb59-4949-ac5b-4e78c7948df7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f15b6902ce664c0ba8fc4cb0868fdaa3', 'request_id': 'req-1b1dc6fa-fb59-4949-ac5b-4e78c7948df7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:24.088790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b1dc6fa-fb59-4949-ac5b-4e78c7948df7 admin admin] The token cache is being invalidate because project f15b6902ce664c0ba8fc4cb0868fdaa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:52:24.089067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b1dc6fa-fb59-4949-ac5b-4e78c7948df7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:24.089727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b1dc6fa-fb59-4949-ac5b-4e78c7948df7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:24.091258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1089/2190] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:52:23 2022] DELETE /identity/v3/projects/f15b6902ce664c0ba8fc4cb0868fdaa3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:52:25.669907 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e856bf-47fd-40c0-87eb-4e6f52480c37 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:25.673529 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d3e856bf-47fd-40c0-87eb-4e6f52480c37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:25.685904 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e856bf-47fd-40c0-87eb-4e6f52480c37 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:25.771920 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db9dfe-d461-4f67-a65f-3981fe4d0db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:25.772958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db9dfe-d461-4f67-a65f-3981fe4d0db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:25.773243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db9dfe-d461-4f67-a65f-3981fe4d0db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:25.773542 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db9dfe-d461-4f67-a65f-3981fe4d0db8 admin admin] PATH_INFO: `/v3/users/ee01546adfef4b2b94a3fc0e47b71edd` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:25.774731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db9dfe-d461-4f67-a65f-3981fe4d0db8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee01546adfef4b2b94a3fc0e47b71edd)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:25.776697 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db9dfe-d461-4f67-a65f-3981fe4d0db8 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:25.828838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-77db9dfe-d461-4f67-a65f-3981fe4d0db8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee01546adfef4b2b94a3fc0e47b71edd'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:25.836278 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-77db9dfe-d461-4f67-a65f-3981fe4d0db8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee01546adfef4b2b94a3fc0e47b71edd'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:25.848846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-77db9dfe-d461-4f67-a65f-3981fe4d0db8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee01546adfef4b2b94a3fc0e47b71edd'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:25.855824 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-77db9dfe-d461-4f67-a65f-3981fe4d0db8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee01546adfef4b2b94a3fc0e47b71edd'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:25.856682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-77db9dfe-d461-4f67-a65f-3981fe4d0db8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee01546adfef4b2b94a3fc0e47b71edd'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:25.863453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1102/2191] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:52:25 2022] DELETE /identity/v3/users/ee01546adfef4b2b94a3fc0e47b71edd => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:52:25.867735 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b46ee4-2008-433d-ba40-adbc6fad73c8 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:25.868195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b46ee4-2008-433d-ba40-adbc6fad73c8 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:25.868315 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b46ee4-2008-433d-ba40-adbc6fad73c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:25.889530 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-94b46ee4-2008-433d-ba40-adbc6fad73c8 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:25.891917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-94b46ee4-2008-433d-ba40-adbc6fad73c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:25.969010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1090/2192] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:52:25.979645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db9dfe-d461-4f67-a65f-3981fe4d0db8 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:25.982519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-77db9dfe-d461-4f67-a65f-3981fe4d0db8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:26.004194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:52:26.063451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db9dfe-d461-4f67-a65f-3981fe4d0db8 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:26.067321 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ff431-70d5-4330-8ce9-0322bd8ac33e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:26.067955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ff431-70d5-4330-8ce9-0322bd8ac33e service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:26.068242 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ff431-70d5-4330-8ce9-0322bd8ac33e service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:26.068594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ff431-70d5-4330-8ce9-0322bd8ac33e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:26.079256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ff431-70d5-4330-8ce9-0322bd8ac33e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:26.080816 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ff431-70d5-4330-8ce9-0322bd8ac33e service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:26.090696 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1103/2193] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:52:26.381786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b46ee4-2008-433d-ba40-adbc6fad73c8 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:26.383413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-94b46ee4-2008-433d-ba40-adbc6fad73c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:26.397152 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b46ee4-2008-433d-ba40-adbc6fad73c8 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:26.403191 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6cd362-f9c6-4b05-95a3-ba365591f232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:26.404019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cd362-f9c6-4b05-95a3-ba365591f232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:26.404476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cd362-f9c6-4b05-95a3-ba365591f232 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:26.404967 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cd362-f9c6-4b05-95a3-ba365591f232 admin admin] PATH_INFO: `/v3/projects/d283268ddb0c4b7b91dbc1bae271c708` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:26.407607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6cd362-f9c6-4b05-95a3-ba365591f232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d283268ddb0c4b7b91dbc1bae271c708)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:26.409902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6cd362-f9c6-4b05-95a3-ba365591f232 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:26.466767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-8d6cd362-f9c6-4b05-95a3-ba365591f232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd283268ddb0c4b7b91dbc1bae271c708', 'request_id': 'req-8d6cd362-f9c6-4b05-95a3-ba365591f232'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:26.474207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-8d6cd362-f9c6-4b05-95a3-ba365591f232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd283268ddb0c4b7b91dbc1bae271c708', 'request_id': 'req-8d6cd362-f9c6-4b05-95a3-ba365591f232'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:26.483405 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-8d6cd362-f9c6-4b05-95a3-ba365591f232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd283268ddb0c4b7b91dbc1bae271c708', 'request_id': 'req-8d6cd362-f9c6-4b05-95a3-ba365591f232'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:26.486217 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:52:26.501049 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-8d6cd362-f9c6-4b05-95a3-ba365591f232 admin admin] The token cache is being invalidate because project d283268ddb0c4b7b91dbc1bae271c708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:52:26.501343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-8d6cd362-f9c6-4b05-95a3-ba365591f232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:26.501989 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-8d6cd362-f9c6-4b05-95a3-ba365591f232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:52:26.503557 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1091/2194] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:52:26 2022] DELETE /identity/v3/projects/d283268ddb0c4b7b91dbc1bae271c708 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:52:26.509792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48a00c-295c-4daa-9ebe-7e6ae9839c8b None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:26.510228 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48a00c-295c-4daa-9ebe-7e6ae9839c8b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:26.510564 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48a00c-295c-4daa-9ebe-7e6ae9839c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:26.535166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-7d48a00c-295c-4daa-9ebe-7e6ae9839c8b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:26.536424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7d48a00c-295c-4daa-9ebe-7e6ae9839c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:26.606128 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1104/2195] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:52:26.614616 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e55b8-0136-472e-b6e1-c99f46054e01 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:26.614906 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e55b8-0136-472e-b6e1-c99f46054e01 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:26.615257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e55b8-0136-472e-b6e1-c99f46054e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:26.636946 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-783e55b8-0136-472e-b6e1-c99f46054e01 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:26.638149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-783e55b8-0136-472e-b6e1-c99f46054e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:26.650248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1092/2196] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:52:26.656009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48a00c-295c-4daa-9ebe-7e6ae9839c8b None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:26.667355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48a00c-295c-4daa-9ebe-7e6ae9839c8b None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:26.672039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efde9e6-0ddd-493a-9772-15118d8d1474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:26.672766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efde9e6-0ddd-493a-9772-15118d8d1474 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:26.672976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efde9e6-0ddd-493a-9772-15118d8d1474 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:26.673249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efde9e6-0ddd-493a-9772-15118d8d1474 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:26.673961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efde9e6-0ddd-493a-9772-15118d8d1474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:52:26.674263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efde9e6-0ddd-493a-9772-15118d8d1474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:26.675879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efde9e6-0ddd-493a-9772-15118d8d1474 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:26.688592 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1105/2197] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:52:26.692994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e55b8-0136-472e-b6e1-c99f46054e01 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:26.703532 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e55b8-0136-472e-b6e1-c99f46054e01 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:26.708444 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bfafc-ec16-4132-85d8-45a66f6f7c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:26.709062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bfafc-ec16-4132-85d8-45a66f6f7c28 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:26.709371 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bfafc-ec16-4132-85d8-45a66f6f7c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:26.709643 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bfafc-ec16-4132-85d8-45a66f6f7c28 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:26.710322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bfafc-ec16-4132-85d8-45a66f6f7c28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:26.712247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bfafc-ec16-4132-85d8-45a66f6f7c28 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:26.742145 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1093/2198] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:52:26 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:52:26.746283 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efde9e6-0ddd-493a-9772-15118d8d1474 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:26.757596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efde9e6-0ddd-493a-9772-15118d8d1474 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:26.859530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1f11c-bf21-4580-b907-ba2327d2a9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:26.860214 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1f11c-bf21-4580-b907-ba2327d2a9fb admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:26.861115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1f11c-bf21-4580-b907-ba2327d2a9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:26.861115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1f11c-bf21-4580-b907-ba2327d2a9fb admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:26.861622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1f11c-bf21-4580-b907-ba2327d2a9fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:26.863588 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1f11c-bf21-4580-b907-ba2327d2a9fb admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:26.895439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1106/2199] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:52:26 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:52:26.899536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bfafc-ec16-4132-85d8-45a66f6f7c28 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:26.917189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bfafc-ec16-4132-85d8-45a66f6f7c28 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:26.927384 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220bc0f-b5ae-4f53-9405-bc283ee1a48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:26.928220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220bc0f-b5ae-4f53-9405-bc283ee1a48d admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:26.928630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220bc0f-b5ae-4f53-9405-bc283ee1a48d admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:26.934503 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220bc0f-b5ae-4f53-9405-bc283ee1a48d admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:26.935623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220bc0f-b5ae-4f53-9405-bc283ee1a48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:26.938726 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220bc0f-b5ae-4f53-9405-bc283ee1a48d admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:26.959547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1094/2200] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:52:26.961335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1f11c-bf21-4580-b907-ba2327d2a9fb admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:26.973643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1f11c-bf21-4580-b907-ba2327d2a9fb admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:26.979194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125b877-1fe0-47ca-b7bc-d6a33c79992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:26.979919 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125b877-1fe0-47ca-b7bc-d6a33c79992d admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:26.980271 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125b877-1fe0-47ca-b7bc-d6a33c79992d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:26.980645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125b877-1fe0-47ca-b7bc-d6a33c79992d admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:26.981469 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125b877-1fe0-47ca-b7bc-d6a33c79992d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:26.983506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125b877-1fe0-47ca-b7bc-d6a33c79992d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:26.997142 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1107/2201] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:52:27.002804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220bc0f-b5ae-4f53-9405-bc283ee1a48d admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:27.017201 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220bc0f-b5ae-4f53-9405-bc283ee1a48d admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:27.025847 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9f79a-b60c-40c2-9fd9-3c40a4dab8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:27.026696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9f79a-b60c-40c2-9fd9-3c40a4dab8f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:27.026905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9f79a-b60c-40c2-9fd9-3c40a4dab8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:27.027188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9f79a-b60c-40c2-9fd9-3c40a4dab8f4 admin admin] PATH_INFO: `/v3/projects/653198c47b6a4f68b5ac4932b02b95d0/users/fe100dded4484be3981b33384f4c68b2/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:27.054216 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9f79a-b60c-40c2-9fd9-3c40a4dab8f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=653198c47b6a4f68b5ac4932b02b95d0, user_id=fe100dded4484be3981b33384f4c68b2, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:27.056993 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9f79a-b60c-40c2-9fd9-3c40a4dab8f4 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:27.073468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1095/2202] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:52:27 2022] PUT /identity/v3/projects/653198c47b6a4f68b5ac4932b02b95d0/users/fe100dded4484be3981b33384f4c68b2/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:52:27.079597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15eb60-6af3-42ed-b465-6d6a751a45e6 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:27.079597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15eb60-6af3-42ed-b465-6d6a751a45e6 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:27.079855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15eb60-6af3-42ed-b465-6d6a751a45e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:27.100723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-cd15eb60-6af3-42ed-b465-6d6a751a45e6 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:27.101932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-cd15eb60-6af3-42ed-b465-6d6a751a45e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:27.117252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:52:27.186091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1108/2203] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:52:27.196602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9f79a-b60c-40c2-9fd9-3c40a4dab8f4 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:27.198445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8dc9f79a-b60c-40c2-9fd9-3c40a4dab8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:27.284370 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9f79a-b60c-40c2-9fd9-3c40a4dab8f4 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:27.288460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6e8e2-d3dc-480f-9f6f-2a982ee09858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:27.289163 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6e8e2-d3dc-480f-9f6f-2a982ee09858 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:27.289519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6e8e2-d3dc-480f-9f6f-2a982ee09858 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:27.289872 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6e8e2-d3dc-480f-9f6f-2a982ee09858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:27.299948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6e8e2-d3dc-480f-9f6f-2a982ee09858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:27.301512 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6e8e2-d3dc-480f-9f6f-2a982ee09858 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:27.312966 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1096/2204] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:52:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:52:28.001222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38118f5-757c-4b2d-9371-2206d28b385c None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:28.001725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38118f5-757c-4b2d-9371-2206d28b385c None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:28.001840 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38118f5-757c-4b2d-9371-2206d28b385c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:28.032104 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-b38118f5-757c-4b2d-9371-2206d28b385c None None] MFA Rules not processed for user `fe100dded4484be3981b33384f4c68b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:28.033400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b38118f5-757c-4b2d-9371-2206d28b385c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:28.110013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1109/2205] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:52:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:52:28.124763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6e8e2-d3dc-480f-9f6f-2a982ee09858 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:52:28.126604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-bdb6e8e2-d3dc-480f-9f6f-2a982ee09858 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:28.197598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6e8e2-d3dc-480f-9f6f-2a982ee09858 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:52:28.201477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f4cdb-9a40-44f9-be5e-2fe160f6a443 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:52:28.202776 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f4cdb-9a40-44f9-be5e-2fe160f6a443 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:28.203229 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f4cdb-9a40-44f9-be5e-2fe160f6a443 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:28.203477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f4cdb-9a40-44f9-be5e-2fe160f6a443 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:28.213746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f4cdb-9a40-44f9-be5e-2fe160f6a443 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:52:28.215310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f4cdb-9a40-44f9-be5e-2fe160f6a443 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:52:28.226489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1097/2206] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:52:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:52:29.645988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10224735-8425-4f52-8382-1b32c0d39016 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:29.646496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10224735-8425-4f52-8382-1b32c0d39016 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:29.646589 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10224735-8425-4f52-8382-1b32c0d39016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:29.667915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-10224735-8425-4f52-8382-1b32c0d39016 None None] MFA Rules not processed for user `fe100dded4484be3981b33384f4c68b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:29.669401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-10224735-8425-4f52-8382-1b32c0d39016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:29.678547 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1110/2207] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:52:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:52:36.498709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c74e6-8109-4d01-a0c2-cb1aed1d6c5f None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:52:36.499253 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c74e6-8109-4d01-a0c2-cb1aed1d6c5f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:52:36.500000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c74e6-8109-4d01-a0c2-cb1aed1d6c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:52:36.520443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-684c74e6-8109-4d01-a0c2-cb1aed1d6c5f None None] MFA Rules not processed for user `9201a1d107bf41959751e7d7c4e4d37d`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:52:36.521661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-684c74e6-8109-4d01-a0c2-cb1aed1d6c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:52:36.550885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:52:36.581288 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1098/2208] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:52:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:52:39.667492 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=53 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:52:46.574533 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=10 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:04.317071 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10224735-8425-4f52-8382-1b32c0d39016 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:04.318614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-10224735-8425-4f52-8382-1b32c0d39016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:04.334119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10224735-8425-4f52-8382-1b32c0d39016 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:04.340111 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1f0c7-d40a-4cfd-b9f1-4ed21d69ed49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:04.340853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1f0c7-d40a-4cfd-b9f1-4ed21d69ed49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:04.341146 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1f0c7-d40a-4cfd-b9f1-4ed21d69ed49 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:04.341430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1f0c7-d40a-4cfd-b9f1-4ed21d69ed49 admin admin] PATH_INFO: `/v3/users/f366266902094cd49554b5b0459272fc` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:04.342660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1f0c7-d40a-4cfd-b9f1-4ed21d69ed49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f366266902094cd49554b5b0459272fc)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:04.344798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1f0c7-d40a-4cfd-b9f1-4ed21d69ed49 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:04.416597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e5d1f0c7-d40a-4cfd-b9f1-4ed21d69ed49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f366266902094cd49554b5b0459272fc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:04.427559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e5d1f0c7-d40a-4cfd-b9f1-4ed21d69ed49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f366266902094cd49554b5b0459272fc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:04.448394 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e5d1f0c7-d40a-4cfd-b9f1-4ed21d69ed49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f366266902094cd49554b5b0459272fc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:04.460143 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e5d1f0c7-d40a-4cfd-b9f1-4ed21d69ed49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f366266902094cd49554b5b0459272fc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:04.460952 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e5d1f0c7-d40a-4cfd-b9f1-4ed21d69ed49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f366266902094cd49554b5b0459272fc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:04.483793 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1111/2209] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:53:04 2022] DELETE /identity/v3/users/f366266902094cd49554b5b0459272fc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:04.489130 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495d935-26bd-468e-9f1c-004d47ad1594 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:04.489654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495d935-26bd-468e-9f1c-004d47ad1594 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:04.489767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495d935-26bd-468e-9f1c-004d47ad1594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:04.513750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-8495d935-26bd-468e-9f1c-004d47ad1594 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:04.514983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8495d935-26bd-468e-9f1c-004d47ad1594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:04.578210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1099/2210] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:04.590128 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1f0c7-d40a-4cfd-b9f1-4ed21d69ed49 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:04.591534 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e5d1f0c7-d40a-4cfd-b9f1-4ed21d69ed49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:04.653675 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1f0c7-d40a-4cfd-b9f1-4ed21d69ed49 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:04.657390 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1d338-6a97-43d8-b7fc-0b8a36a0031b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:04.658761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1d338-6a97-43d8-b7fc-0b8a36a0031b service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:04.658970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1d338-6a97-43d8-b7fc-0b8a36a0031b service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:04.659226 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1d338-6a97-43d8-b7fc-0b8a36a0031b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:04.668850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1d338-6a97-43d8-b7fc-0b8a36a0031b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:04.670282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1d338-6a97-43d8-b7fc-0b8a36a0031b service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:04.680623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1112/2211] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:53:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:53:04.957411 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495d935-26bd-468e-9f1c-004d47ad1594 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:04.958840 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8495d935-26bd-468e-9f1c-004d47ad1594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:04.975596 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495d935-26bd-468e-9f1c-004d47ad1594 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:04.980334 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:04.981007 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:04.981298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:04.981576 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c admin admin] PATH_INFO: `/v3/projects/5b71821ba42d4b9987a6fc6943c4f817` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:04.982761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b71821ba42d4b9987a6fc6943c4f817)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:04.984987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:05.037995 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b71821ba42d4b9987a6fc6943c4f817', 'request_id': 'req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:05.043788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b71821ba42d4b9987a6fc6943c4f817', 'request_id': 'req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:05.050944 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b71821ba42d4b9987a6fc6943c4f817', 'request_id': 'req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:05.066536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c admin admin] The token cache is being invalidate because project 5b71821ba42d4b9987a6fc6943c4f817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:53:05.066829 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:05.067487 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:05.069053 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1100/2212] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:53:04 2022] DELETE /identity/v3/projects/5b71821ba42d4b9987a6fc6943c4f817 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:05.073034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1d338-6a97-43d8-b7fc-0b8a36a0031b service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:05.084006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1d338-6a97-43d8-b7fc-0b8a36a0031b service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:05.150692 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6e279-5bf8-4475-87f7-77109551f3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:05.151250 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6e279-5bf8-4475-87f7-77109551f3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:05.151531 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6e279-5bf8-4475-87f7-77109551f3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:05.151805 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6e279-5bf8-4475-87f7-77109551f3b0 admin admin] PATH_INFO: `/v3/users/226fde94e62c4d9394846179aa2d0cd6` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:05.153000 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6e279-5bf8-4475-87f7-77109551f3b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=226fde94e62c4d9394846179aa2d0cd6)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:05.155557 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6e279-5bf8-4475-87f7-77109551f3b0 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:05.174900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:05.211363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5bc6e279-5bf8-4475-87f7-77109551f3b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226fde94e62c4d9394846179aa2d0cd6'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:05.219363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5bc6e279-5bf8-4475-87f7-77109551f3b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226fde94e62c4d9394846179aa2d0cd6'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:05.232658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5bc6e279-5bf8-4475-87f7-77109551f3b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226fde94e62c4d9394846179aa2d0cd6'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:05.241924 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5bc6e279-5bf8-4475-87f7-77109551f3b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226fde94e62c4d9394846179aa2d0cd6'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:05.242825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5bc6e279-5bf8-4475-87f7-77109551f3b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226fde94e62c4d9394846179aa2d0cd6'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:05.250948 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1113/2213] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:53:05 2022] DELETE /identity/v3/users/226fde94e62c4d9394846179aa2d0cd6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:53:05.558841 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:05.560506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:05.574936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e2427-70ce-43e9-9c30-fb04d28ea84c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:05.645606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac86d04-3611-447b-94c5-a0e3fd69fb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:05.646462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac86d04-3611-447b-94c5-a0e3fd69fb5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:05.646755 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac86d04-3611-447b-94c5-a0e3fd69fb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:05.647077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac86d04-3611-447b-94c5-a0e3fd69fb5d admin admin] PATH_INFO: `/v3/projects/d7204dc12c324849afab1defe67f0fd7` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:05.648313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac86d04-3611-447b-94c5-a0e3fd69fb5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7204dc12c324849afab1defe67f0fd7)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:05.650266 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac86d04-3611-447b-94c5-a0e3fd69fb5d admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:05.685103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:05.708750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ac86d04-3611-447b-94c5-a0e3fd69fb5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7204dc12c324849afab1defe67f0fd7', 'request_id': 'req-5ac86d04-3611-447b-94c5-a0e3fd69fb5d'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:05.714312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ac86d04-3611-447b-94c5-a0e3fd69fb5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7204dc12c324849afab1defe67f0fd7', 'request_id': 'req-5ac86d04-3611-447b-94c5-a0e3fd69fb5d'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:05.721054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ac86d04-3611-447b-94c5-a0e3fd69fb5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7204dc12c324849afab1defe67f0fd7', 'request_id': 'req-5ac86d04-3611-447b-94c5-a0e3fd69fb5d'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:05.737706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ac86d04-3611-447b-94c5-a0e3fd69fb5d admin admin] The token cache is being invalidate because project d7204dc12c324849afab1defe67f0fd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:53:05.738000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ac86d04-3611-447b-94c5-a0e3fd69fb5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:05.738654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ac86d04-3611-447b-94c5-a0e3fd69fb5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:05.740617 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1101/2214] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:53:05 2022] DELETE /identity/v3/projects/d7204dc12c324849afab1defe67f0fd7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:05.746140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d302407-a3e4-4cb8-b93c-1bb908f7e2ec None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:05.746635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d302407-a3e4-4cb8-b93c-1bb908f7e2ec None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:05.746728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d302407-a3e4-4cb8-b93c-1bb908f7e2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:05.769927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-6d302407-a3e4-4cb8-b93c-1bb908f7e2ec None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:05.771148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-6d302407-a3e4-4cb8-b93c-1bb908f7e2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:05.834682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1114/2215] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:05.844123 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a7bc9-e17d-467f-929a-7854e4dbd812 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:05.844439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a7bc9-e17d-467f-929a-7854e4dbd812 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:05.844722 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a7bc9-e17d-467f-929a-7854e4dbd812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:05.865427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-850a7bc9-e17d-467f-929a-7854e4dbd812 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:05.866933 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-850a7bc9-e17d-467f-929a-7854e4dbd812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:05.880576 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1102/2216] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:05.882346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d302407-a3e4-4cb8-b93c-1bb908f7e2ec None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:05.901709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d302407-a3e4-4cb8-b93c-1bb908f7e2ec None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:05.908640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487085b-baf5-4bad-bd66-d5611a55bbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:05.909309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487085b-baf5-4bad-bd66-d5611a55bbc5 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:05.910304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487085b-baf5-4bad-bd66-d5611a55bbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:05.910586 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487085b-baf5-4bad-bd66-d5611a55bbc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:05.911483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487085b-baf5-4bad-bd66-d5611a55bbc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:53:05.912582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487085b-baf5-4bad-bd66-d5611a55bbc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:05.917543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487085b-baf5-4bad-bd66-d5611a55bbc5 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:05.933135 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1115/2217] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:53:05.937594 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a7bc9-e17d-467f-929a-7854e4dbd812 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:05.949669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a7bc9-e17d-467f-929a-7854e4dbd812 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:05.955273 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c636e98-f6c5-4b21-98e8-ad03b48f2297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:05.955967 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c636e98-f6c5-4b21-98e8-ad03b48f2297 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:05.956259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c636e98-f6c5-4b21-98e8-ad03b48f2297 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:05.956566 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c636e98-f6c5-4b21-98e8-ad03b48f2297 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:05.957422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c636e98-f6c5-4b21-98e8-ad03b48f2297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:05.959420 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c636e98-f6c5-4b21-98e8-ad03b48f2297 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:05.996341 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1103/2218] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:53:05 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:53:06.002334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487085b-baf5-4bad-bd66-d5611a55bbc5 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:06.014158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487085b-baf5-4bad-bd66-d5611a55bbc5 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:06.075707 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb18665-de8c-4b19-8537-2853c577241d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:06.076543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb18665-de8c-4b19-8537-2853c577241d admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:06.076742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb18665-de8c-4b19-8537-2853c577241d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:06.077283 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb18665-de8c-4b19-8537-2853c577241d admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:06.077853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb18665-de8c-4b19-8537-2853c577241d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:06.079779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb18665-de8c-4b19-8537-2853c577241d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:06.100835 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1116/2219] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:53:06 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:53:06.104917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c636e98-f6c5-4b21-98e8-ad03b48f2297 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:06.116316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c636e98-f6c5-4b21-98e8-ad03b48f2297 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:06.120870 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fb84b-d102-483d-acc0-a4075707964f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:06.121499 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fb84b-d102-483d-acc0-a4075707964f admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:06.121816 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fb84b-d102-483d-acc0-a4075707964f admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:06.122045 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fb84b-d102-483d-acc0-a4075707964f admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:06.122746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1fb84b-d102-483d-acc0-a4075707964f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:06.124354 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1fb84b-d102-483d-acc0-a4075707964f admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:06.134471 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1104/2220] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:53:06.138425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb18665-de8c-4b19-8537-2853c577241d admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:06.148888 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb18665-de8c-4b19-8537-2853c577241d admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:06.153569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e222c37-70b6-4afe-a0c8-07a83bd90f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:06.154199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e222c37-70b6-4afe-a0c8-07a83bd90f22 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:06.154475 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e222c37-70b6-4afe-a0c8-07a83bd90f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:06.154752 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e222c37-70b6-4afe-a0c8-07a83bd90f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:06.155449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e222c37-70b6-4afe-a0c8-07a83bd90f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:06.157090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e222c37-70b6-4afe-a0c8-07a83bd90f22 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:06.163317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:06.169508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1117/2221] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:53:06.174245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fb84b-d102-483d-acc0-a4075707964f admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:06.184227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fb84b-d102-483d-acc0-a4075707964f admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:06.188954 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325bc95-47e6-4b70-8abc-1acab5caa399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:06.189770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325bc95-47e6-4b70-8abc-1acab5caa399 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:06.190058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325bc95-47e6-4b70-8abc-1acab5caa399 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:06.190339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325bc95-47e6-4b70-8abc-1acab5caa399 admin admin] PATH_INFO: `/v3/projects/ef67fc14c4da445cadc333551939c888/users/478129a754b1456aa6148956e430b4c6/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:06.205023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325bc95-47e6-4b70-8abc-1acab5caa399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef67fc14c4da445cadc333551939c888, user_id=478129a754b1456aa6148956e430b4c6, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:06.207701 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325bc95-47e6-4b70-8abc-1acab5caa399 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:06.226124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1105/2222] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:53:06 2022] PUT /identity/v3/projects/ef67fc14c4da445cadc333551939c888/users/478129a754b1456aa6148956e430b4c6/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:53:06.238131 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e479e39-4459-4016-bbd4-82182e29d8e0 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:06.238413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e479e39-4459-4016-bbd4-82182e29d8e0 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:06.238686 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e479e39-4459-4016-bbd4-82182e29d8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:06.270561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-9e479e39-4459-4016-bbd4-82182e29d8e0 None None] MFA Rules not processed for user `478129a754b1456aa6148956e430b4c6`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:06.271775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-9e479e39-4459-4016-bbd4-82182e29d8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:06.344399 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1118/2223] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:06.349376 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325bc95-47e6-4b70-8abc-1acab5caa399 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:06.415859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325bc95-47e6-4b70-8abc-1acab5caa399 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:06.421014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506a7a3-046c-4d4a-b356-ea7676c0c00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:06.421649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506a7a3-046c-4d4a-b356-ea7676c0c00e admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:06.421939 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506a7a3-046c-4d4a-b356-ea7676c0c00e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:06.422214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506a7a3-046c-4d4a-b356-ea7676c0c00e admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:06.422936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506a7a3-046c-4d4a-b356-ea7676c0c00e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:06.425550 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506a7a3-046c-4d4a-b356-ea7676c0c00e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:06.457794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1106/2224] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:53:06 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:53:06.459369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e479e39-4459-4016-bbd4-82182e29d8e0 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:06.471279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e479e39-4459-4016-bbd4-82182e29d8e0 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:06.531413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de3940-3d64-475d-b129-13b37144972a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:06.532090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de3940-3d64-475d-b129-13b37144972a admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:06.532407 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de3940-3d64-475d-b129-13b37144972a admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:06.533789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de3940-3d64-475d-b129-13b37144972a admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:06.534017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de3940-3d64-475d-b129-13b37144972a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:06.535859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de3940-3d64-475d-b129-13b37144972a admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:06.559010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1119/2225] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:53:06 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:53:06.563119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506a7a3-046c-4d4a-b356-ea7676c0c00e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:06.574017 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506a7a3-046c-4d4a-b356-ea7676c0c00e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:06.579157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692b80c-45de-4212-afc6-bc3ef9e461a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:06.580553 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692b80c-45de-4212-afc6-bc3ef9e461a8 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:06.580945 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692b80c-45de-4212-afc6-bc3ef9e461a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:06.581228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692b80c-45de-4212-afc6-bc3ef9e461a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:06.581933 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692b80c-45de-4212-afc6-bc3ef9e461a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:06.583852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692b80c-45de-4212-afc6-bc3ef9e461a8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:06.604416 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1107/2226] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:53:06.608421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de3940-3d64-475d-b129-13b37144972a admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:06.619096 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de3940-3d64-475d-b129-13b37144972a admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:06.623822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed38577-1511-4bbf-833b-1ca86a74b28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:06.624467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed38577-1511-4bbf-833b-1ca86a74b28a admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:06.624747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed38577-1511-4bbf-833b-1ca86a74b28a admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:06.625022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed38577-1511-4bbf-833b-1ca86a74b28a admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:06.625713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed38577-1511-4bbf-833b-1ca86a74b28a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:06.627846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed38577-1511-4bbf-833b-1ca86a74b28a admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:06.638289 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1120/2227] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:53:06.643267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692b80c-45de-4212-afc6-bc3ef9e461a8 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:06.653300 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692b80c-45de-4212-afc6-bc3ef9e461a8 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:06.658519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d26fc-ebc7-4256-9201-d8c4504ca69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:06.659184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d26fc-ebc7-4256-9201-d8c4504ca69e admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:06.659468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d26fc-ebc7-4256-9201-d8c4504ca69e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:06.659742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d26fc-ebc7-4256-9201-d8c4504ca69e admin admin] PATH_INFO: `/v3/projects/e5ef9041d7624063b570ecf35b5da2bd/users/e68cb3f6269b4571b60044c2be5801ed/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:06.677208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d26fc-ebc7-4256-9201-d8c4504ca69e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ef9041d7624063b570ecf35b5da2bd, user_id=e68cb3f6269b4571b60044c2be5801ed, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:06.679896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d26fc-ebc7-4256-9201-d8c4504ca69e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:06.689830 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:06.695533 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1108/2228] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:53:06 2022] PUT /identity/v3/projects/e5ef9041d7624063b570ecf35b5da2bd/users/e68cb3f6269b4571b60044c2be5801ed/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:53:06.706553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b12845-1688-440b-89a6-1983ddcd775a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:06.706874 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b12845-1688-440b-89a6-1983ddcd775a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:06.707115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b12845-1688-440b-89a6-1983ddcd775a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:06.738666 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-b2b12845-1688-440b-89a6-1983ddcd775a None None] MFA Rules not processed for user `e68cb3f6269b4571b60044c2be5801ed`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:06.740144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b2b12845-1688-440b-89a6-1983ddcd775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:06.803892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:06.810299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1121/2229] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:06.815639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d26fc-ebc7-4256-9201-d8c4504ca69e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:06.880136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d26fc-ebc7-4256-9201-d8c4504ca69e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:06.884808 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e89f58-ef98-46a1-812c-0053b2a9b094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:06.885419 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e89f58-ef98-46a1-812c-0053b2a9b094 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:06.885701 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e89f58-ef98-46a1-812c-0053b2a9b094 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:06.885976 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e89f58-ef98-46a1-812c-0053b2a9b094 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:06.886643 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e89f58-ef98-46a1-812c-0053b2a9b094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:06.888576 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e89f58-ef98-46a1-812c-0053b2a9b094 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:06.919923 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1109/2230] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:53:06 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:53:06.921073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b12845-1688-440b-89a6-1983ddcd775a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:06.931832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b12845-1688-440b-89a6-1983ddcd775a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:06.994290 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe32acd-541d-4fdd-b7da-8061e33f4ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:06.994933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe32acd-541d-4fdd-b7da-8061e33f4ede admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:06.995215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe32acd-541d-4fdd-b7da-8061e33f4ede admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:06.995484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe32acd-541d-4fdd-b7da-8061e33f4ede admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:06.996175 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe32acd-541d-4fdd-b7da-8061e33f4ede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:06.998402 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe32acd-541d-4fdd-b7da-8061e33f4ede admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:07.026993 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1122/2231] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:53:06 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:53:07.028435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e89f58-ef98-46a1-812c-0053b2a9b094 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:07.045619 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e89f58-ef98-46a1-812c-0053b2a9b094 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:07.053375 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99092e29-84d5-4573-ae46-dcafaae46c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:07.054380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99092e29-84d5-4573-ae46-dcafaae46c5e admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:07.054380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99092e29-84d5-4573-ae46-dcafaae46c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:07.054619 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99092e29-84d5-4573-ae46-dcafaae46c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:07.055317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99092e29-84d5-4573-ae46-dcafaae46c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:07.057091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99092e29-84d5-4573-ae46-dcafaae46c5e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:07.074446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1110/2232] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:53:07.077409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe32acd-541d-4fdd-b7da-8061e33f4ede admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:07.088804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe32acd-541d-4fdd-b7da-8061e33f4ede admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:07.093433 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65addcdc-6c5a-480b-bccc-c0190c851805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:07.094108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65addcdc-6c5a-480b-bccc-c0190c851805 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:07.094392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65addcdc-6c5a-480b-bccc-c0190c851805 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:07.094670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65addcdc-6c5a-480b-bccc-c0190c851805 admin admin] PATH_INFO: `/v3/projects/033aff263bbb49739b137631d6f471c2/users/f38927c78a7a4a6696a94dd9e65a2f58/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:07.119857 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65addcdc-6c5a-480b-bccc-c0190c851805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=033aff263bbb49739b137631d6f471c2, user_id=f38927c78a7a4a6696a94dd9e65a2f58, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:07.122583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65addcdc-6c5a-480b-bccc-c0190c851805 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:07.137043 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1123/2233] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:53:07 2022] PUT /identity/v3/projects/033aff263bbb49739b137631d6f471c2/users/f38927c78a7a4a6696a94dd9e65a2f58/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:07.141413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99092e29-84d5-4573-ae46-dcafaae46c5e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:07.152276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99092e29-84d5-4573-ae46-dcafaae46c5e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:07.218672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85527335-77a6-4c3f-9edb-84a2db6044b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:07.219341 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85527335-77a6-4c3f-9edb-84a2db6044b3 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:07.219627 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85527335-77a6-4c3f-9edb-84a2db6044b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:07.219908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85527335-77a6-4c3f-9edb-84a2db6044b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:07.220787 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85527335-77a6-4c3f-9edb-84a2db6044b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:07.222574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85527335-77a6-4c3f-9edb-84a2db6044b3 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:07.237564 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1111/2234] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:53:07.238833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65addcdc-6c5a-480b-bccc-c0190c851805 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:07.250502 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65addcdc-6c5a-480b-bccc-c0190c851805 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:07.257269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1820167-9280-44a7-be04-1ad2c7428c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:07.257952 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1820167-9280-44a7-be04-1ad2c7428c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:07.258239 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1820167-9280-44a7-be04-1ad2c7428c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:07.258507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1820167-9280-44a7-be04-1ad2c7428c0e admin admin] PATH_INFO: `/v3/projects/033aff263bbb49739b137631d6f471c2/users/f38927c78a7a4a6696a94dd9e65a2f58/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:07.260551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1820167-9280-44a7-be04-1ad2c7428c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=033aff263bbb49739b137631d6f471c2, user_id=f38927c78a7a4a6696a94dd9e65a2f58, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:07.263198 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1820167-9280-44a7-be04-1ad2c7428c0e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:07.275601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1124/2235] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:53:07 2022] PUT /identity/v3/projects/033aff263bbb49739b137631d6f471c2/users/f38927c78a7a4a6696a94dd9e65a2f58/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:07.287517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6c2b7-2cd4-4431-9cb0-8bb1ef1ce2de None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:07.288050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6c2b7-2cd4-4431-9cb0-8bb1ef1ce2de None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:07.288226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6c2b7-2cd4-4431-9cb0-8bb1ef1ce2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:07.320091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f3b6c2b7-2cd4-4431-9cb0-8bb1ef1ce2de None None] MFA Rules not processed for user `f38927c78a7a4a6696a94dd9e65a2f58`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:07.321400 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f3b6c2b7-2cd4-4431-9cb0-8bb1ef1ce2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:07.414921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:07.428658 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1112/2236] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:07.446515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1820167-9280-44a7-be04-1ad2c7428c0e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:07.448237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a1820167-9280-44a7-be04-1ad2c7428c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:07.512922 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1820167-9280-44a7-be04-1ad2c7428c0e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:07.516559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f44d0-4cf9-4827-8f2f-795f585b2633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:07.517828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f44d0-4cf9-4827-8f2f-795f585b2633 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:07.518224 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f44d0-4cf9-4827-8f2f-795f585b2633 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:07.518623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f44d0-4cf9-4827-8f2f-795f585b2633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:07.528834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f44d0-4cf9-4827-8f2f-795f585b2633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:07.530535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f44d0-4cf9-4827-8f2f-795f585b2633 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:07.542081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1125/2237] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:53:07 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:53:07.590671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6c2b7-2cd4-4431-9cb0-8bb1ef1ce2de None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:07.592160 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f3b6c2b7-2cd4-4431-9cb0-8bb1ef1ce2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:07.617692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6c2b7-2cd4-4431-9cb0-8bb1ef1ce2de None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:07.641762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a187ea3-ee13-4c92-8c87-73aef6311464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:07.643303 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a187ea3-ee13-4c92-8c87-73aef6311464 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:07.643938 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a187ea3-ee13-4c92-8c87-73aef6311464 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:07.644299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a187ea3-ee13-4c92-8c87-73aef6311464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:07.721351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a187ea3-ee13-4c92-8c87-73aef6311464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:07.722845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a187ea3-ee13-4c92-8c87-73aef6311464 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:07.732867 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1113/2238] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:53:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:53:08.672816 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144a592-9e72-4898-bf66-2aa7509d4785 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:08.673999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144a592-9e72-4898-bf66-2aa7509d4785 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:08.674280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144a592-9e72-4898-bf66-2aa7509d4785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:08.695638 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-2144a592-9e72-4898-bf66-2aa7509d4785 None None] MFA Rules not processed for user `c6085ba9dff1480a8a1674e492e0c6c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:08.696866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2144a592-9e72-4898-bf66-2aa7509d4785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:08.746940 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:08.756976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1126/2239] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:53:11.801852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a187ea3-ee13-4c92-8c87-73aef6311464 service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:11.873704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a187ea3-ee13-4c92-8c87-73aef6311464 service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:11.882043 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2efa74-1e85-48be-91ce-b5ac470e39ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:11.882700 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2efa74-1e85-48be-91ce-b5ac470e39ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:11.883165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2efa74-1e85-48be-91ce-b5ac470e39ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:11.883445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2efa74-1e85-48be-91ce-b5ac470e39ee admin admin] PATH_INFO: `/v3/users/478129a754b1456aa6148956e430b4c6` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:11.885041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2efa74-1e85-48be-91ce-b5ac470e39ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=478129a754b1456aa6148956e430b4c6)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:11.887881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2efa74-1e85-48be-91ce-b5ac470e39ee admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:11.958444 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9e2efa74-1e85-48be-91ce-b5ac470e39ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '478129a754b1456aa6148956e430b4c6'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:11.966711 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9e2efa74-1e85-48be-91ce-b5ac470e39ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '478129a754b1456aa6148956e430b4c6'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:11.981328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9e2efa74-1e85-48be-91ce-b5ac470e39ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '478129a754b1456aa6148956e430b4c6'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:11.994027 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9e2efa74-1e85-48be-91ce-b5ac470e39ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '478129a754b1456aa6148956e430b4c6'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:11.994898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9e2efa74-1e85-48be-91ce-b5ac470e39ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '478129a754b1456aa6148956e430b4c6'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:12.006289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1114/2240] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:53:11 2022] DELETE /identity/v3/users/478129a754b1456aa6148956e430b4c6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:12.011599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57c9cb-aa42-4998-a639-9e979da187d9 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:12.011599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57c9cb-aa42-4998-a639-9e979da187d9 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:12.012202 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57c9cb-aa42-4998-a639-9e979da187d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:12.038379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-4b57c9cb-aa42-4998-a639-9e979da187d9 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:12.039611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4b57c9cb-aa42-4998-a639-9e979da187d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:12.155050 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1127/2241] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:12.162530 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2efa74-1e85-48be-91ce-b5ac470e39ee admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:12.163998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9e2efa74-1e85-48be-91ce-b5ac470e39ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:12.181545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:12.282325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2efa74-1e85-48be-91ce-b5ac470e39ee admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:12.286209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982362ef-88f6-4fe4-bbc0-7663cc2b122a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:12.286855 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982362ef-88f6-4fe4-bbc0-7663cc2b122a service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:12.287131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982362ef-88f6-4fe4-bbc0-7663cc2b122a service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:12.287412 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982362ef-88f6-4fe4-bbc0-7663cc2b122a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:12.300313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982362ef-88f6-4fe4-bbc0-7663cc2b122a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:12.302331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982362ef-88f6-4fe4-bbc0-7663cc2b122a service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:12.317691 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1115/2242] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:53:12.680453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57c9cb-aa42-4998-a639-9e979da187d9 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:12.682260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4b57c9cb-aa42-4998-a639-9e979da187d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:12.701391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57c9cb-aa42-4998-a639-9e979da187d9 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:12.710192 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:12.711059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:12.711370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:12.711615 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d admin admin] PATH_INFO: `/v3/projects/ef67fc14c4da445cadc333551939c888` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:12.713541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef67fc14c4da445cadc333551939c888)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:12.716582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:12.792609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef67fc14c4da445cadc333551939c888', 'request_id': 'req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:12.801314 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef67fc14c4da445cadc333551939c888', 'request_id': 'req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:12.816893 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef67fc14c4da445cadc333551939c888', 'request_id': 'req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:12.842970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d admin admin] The token cache is being invalidate because project ef67fc14c4da445cadc333551939c888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:53:12.843270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:12.843994 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:12.846381 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1128/2243] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:53:12 2022] DELETE /identity/v3/projects/ef67fc14c4da445cadc333551939c888 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:12.850393 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982362ef-88f6-4fe4-bbc0-7663cc2b122a service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:12.865665 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982362ef-88f6-4fe4-bbc0-7663cc2b122a service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:13.007255 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946a106-8e80-41f4-bb12-d83ba7f33cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:13.008024 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946a106-8e80-41f4-bb12-d83ba7f33cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:13.008364 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946a106-8e80-41f4-bb12-d83ba7f33cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:13.008654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946a106-8e80-41f4-bb12-d83ba7f33cf4 admin admin] PATH_INFO: `/v3/users/e68cb3f6269b4571b60044c2be5801ed` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:13.010080 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946a106-8e80-41f4-bb12-d83ba7f33cf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e68cb3f6269b4571b60044c2be5801ed)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:13.012153 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946a106-8e80-41f4-bb12-d83ba7f33cf4 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:13.108800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5946a106-8e80-41f4-bb12-d83ba7f33cf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e68cb3f6269b4571b60044c2be5801ed'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:13.119774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5946a106-8e80-41f4-bb12-d83ba7f33cf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e68cb3f6269b4571b60044c2be5801ed'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:13.133817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5946a106-8e80-41f4-bb12-d83ba7f33cf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e68cb3f6269b4571b60044c2be5801ed'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:13.142204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5946a106-8e80-41f4-bb12-d83ba7f33cf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e68cb3f6269b4571b60044c2be5801ed'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:13.142959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5946a106-8e80-41f4-bb12-d83ba7f33cf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e68cb3f6269b4571b60044c2be5801ed'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:13.153485 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1116/2244] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:53:12 2022] DELETE /identity/v3/users/e68cb3f6269b4571b60044c2be5801ed => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:53:13.593114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:13.594794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:13.609234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585d3fc-da00-4f4c-a1f5-8d2fa7d4383d admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:13.673733 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:13.675113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:13.675413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:13.675851 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501 admin admin] PATH_INFO: `/v3/projects/e5ef9041d7624063b570ecf35b5da2bd` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:13.677077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5ef9041d7624063b570ecf35b5da2bd)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:13.679007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:13.707555 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:13.730693 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5ef9041d7624063b570ecf35b5da2bd', 'request_id': 'req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:13.736244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5ef9041d7624063b570ecf35b5da2bd', 'request_id': 'req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:13.743552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5ef9041d7624063b570ecf35b5da2bd', 'request_id': 'req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:13.755637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501 admin admin] The token cache is being invalidate because project e5ef9041d7624063b570ecf35b5da2bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:53:13.755926 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:13.756576 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:13.758079 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1129/2245] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:53:13 2022] DELETE /identity/v3/projects/e5ef9041d7624063b570ecf35b5da2bd => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:13.762037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946a106-8e80-41f4-bb12-d83ba7f33cf4 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:13.763504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-5946a106-8e80-41f4-bb12-d83ba7f33cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:13.777509 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946a106-8e80-41f4-bb12-d83ba7f33cf4 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:13.811171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:13.840505 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e8d88-dbdc-40ba-b462-07eff8017a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:13.841176 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e8d88-dbdc-40ba-b462-07eff8017a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:13.841458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e8d88-dbdc-40ba-b462-07eff8017a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:13.841731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e8d88-dbdc-40ba-b462-07eff8017a88 admin admin] PATH_INFO: `/v3/users/f38927c78a7a4a6696a94dd9e65a2f58` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:13.842919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e8d88-dbdc-40ba-b462-07eff8017a88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38927c78a7a4a6696a94dd9e65a2f58)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:13.845065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e8d88-dbdc-40ba-b462-07eff8017a88 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:13.894144 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ea7e8d88-dbdc-40ba-b462-07eff8017a88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38927c78a7a4a6696a94dd9e65a2f58'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:13.902340 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ea7e8d88-dbdc-40ba-b462-07eff8017a88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38927c78a7a4a6696a94dd9e65a2f58'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:13.921316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ea7e8d88-dbdc-40ba-b462-07eff8017a88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38927c78a7a4a6696a94dd9e65a2f58'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:13.928852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ea7e8d88-dbdc-40ba-b462-07eff8017a88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38927c78a7a4a6696a94dd9e65a2f58'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:13.929593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ea7e8d88-dbdc-40ba-b462-07eff8017a88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38927c78a7a4a6696a94dd9e65a2f58'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:13.936443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1117/2246] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:53:13 2022] DELETE /identity/v3/users/f38927c78a7a4a6696a94dd9e65a2f58 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:14.268778 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:14.270338 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:14.283206 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbbdaf-cbee-479c-bd64-a8dc4bc55501 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:14.354553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3442a-8206-42a6-9bac-e4a1ee0d4778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:14.355220 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3442a-8206-42a6-9bac-e4a1ee0d4778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:14.355556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3442a-8206-42a6-9bac-e4a1ee0d4778 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:14.355837 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3442a-8206-42a6-9bac-e4a1ee0d4778 admin admin] PATH_INFO: `/v3/projects/033aff263bbb49739b137631d6f471c2` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:14.357293 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3442a-8206-42a6-9bac-e4a1ee0d4778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=033aff263bbb49739b137631d6f471c2)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:14.359268 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3442a-8206-42a6-9bac-e4a1ee0d4778 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:14.412114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-cae3442a-8206-42a6-9bac-e4a1ee0d4778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '033aff263bbb49739b137631d6f471c2', 'request_id': 'req-cae3442a-8206-42a6-9bac-e4a1ee0d4778'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:14.418398 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-cae3442a-8206-42a6-9bac-e4a1ee0d4778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '033aff263bbb49739b137631d6f471c2', 'request_id': 'req-cae3442a-8206-42a6-9bac-e4a1ee0d4778'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:14.427327 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-cae3442a-8206-42a6-9bac-e4a1ee0d4778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '033aff263bbb49739b137631d6f471c2', 'request_id': 'req-cae3442a-8206-42a6-9bac-e4a1ee0d4778'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:14.439922 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-cae3442a-8206-42a6-9bac-e4a1ee0d4778 admin admin] The token cache is being invalidate because project 033aff263bbb49739b137631d6f471c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:53:14.440212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-cae3442a-8206-42a6-9bac-e4a1ee0d4778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:14.440914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-cae3442a-8206-42a6-9bac-e4a1ee0d4778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:14.443136 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1130/2247] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:53:14 2022] DELETE /identity/v3/projects/033aff263bbb49739b137631d6f471c2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:14.449282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c38bdd-b634-4147-bd82-a9c3565d1740 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:14.449574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c38bdd-b634-4147-bd82-a9c3565d1740 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:14.449876 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c38bdd-b634-4147-bd82-a9c3565d1740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:14.472057 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-a4c38bdd-b634-4147-bd82-a9c3565d1740 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:14.473586 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a4c38bdd-b634-4147-bd82-a9c3565d1740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:14.536116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1118/2248] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:53:14.544671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d3642-ab9d-4cf4-9c05-9281b822db80 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:14.545030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d3642-ab9d-4cf4-9c05-9281b822db80 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:14.545377 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d3642-ab9d-4cf4-9c05-9281b822db80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:14.566516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d19d3642-ab9d-4cf4-9c05-9281b822db80 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:14.567786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d19d3642-ab9d-4cf4-9c05-9281b822db80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:14.575867 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1131/2249] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:14.580189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c38bdd-b634-4147-bd82-a9c3565d1740 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:14.590061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c38bdd-b634-4147-bd82-a9c3565d1740 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:14.596337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf2fa2-d59d-441a-a046-f4ff8b259ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:14.597175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf2fa2-d59d-441a-a046-f4ff8b259ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:14.597470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf2fa2-d59d-441a-a046-f4ff8b259ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:14.597730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf2fa2-d59d-441a-a046-f4ff8b259ef7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:14.598443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf2fa2-d59d-441a-a046-f4ff8b259ef7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:53:14.598753 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf2fa2-d59d-441a-a046-f4ff8b259ef7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:14.600461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf2fa2-d59d-441a-a046-f4ff8b259ef7 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:14.613124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1119/2250] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:53:14.617471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d3642-ab9d-4cf4-9c05-9281b822db80 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:14.627223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d3642-ab9d-4cf4-9c05-9281b822db80 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:14.632774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4c45d-038e-4914-b79a-0cecdeccb729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:14.633397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4c45d-038e-4914-b79a-0cecdeccb729 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:14.633677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4c45d-038e-4914-b79a-0cecdeccb729 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:14.633948 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4c45d-038e-4914-b79a-0cecdeccb729 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:14.634628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4c45d-038e-4914-b79a-0cecdeccb729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:14.636774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4c45d-038e-4914-b79a-0cecdeccb729 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:14.665024 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1132/2251] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:53:14 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:53:14.669271 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf2fa2-d59d-441a-a046-f4ff8b259ef7 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:14.679888 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf2fa2-d59d-441a-a046-f4ff8b259ef7 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:14.721402 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:14.740837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68a4ad-aabc-4de5-b9fc-f19041b341dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:14.741509 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68a4ad-aabc-4de5-b9fc-f19041b341dc admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:14.741816 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68a4ad-aabc-4de5-b9fc-f19041b341dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:14.742095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68a4ad-aabc-4de5-b9fc-f19041b341dc admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:14.742765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68a4ad-aabc-4de5-b9fc-f19041b341dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:14.744705 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68a4ad-aabc-4de5-b9fc-f19041b341dc admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:14.766161 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1120/2252] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:53:14 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:53:14.771058 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4c45d-038e-4914-b79a-0cecdeccb729 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:14.781189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4c45d-038e-4914-b79a-0cecdeccb729 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:14.787273 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984a4ed-f311-446f-9d9f-49fd331e3b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:14.787882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984a4ed-f311-446f-9d9f-49fd331e3b9b admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:14.788159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984a4ed-f311-446f-9d9f-49fd331e3b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:14.788471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984a4ed-f311-446f-9d9f-49fd331e3b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:14.789160 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984a4ed-f311-446f-9d9f-49fd331e3b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:14.790749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984a4ed-f311-446f-9d9f-49fd331e3b9b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:14.801072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1133/2253] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:53:14.805229 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68a4ad-aabc-4de5-b9fc-f19041b341dc admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:14.815950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68a4ad-aabc-4de5-b9fc-f19041b341dc admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:14.820694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5306a0-fa02-427e-8888-9ec4a8546a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:14.821306 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5306a0-fa02-427e-8888-9ec4a8546a21 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:14.821591 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5306a0-fa02-427e-8888-9ec4a8546a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:14.821873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5306a0-fa02-427e-8888-9ec4a8546a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:14.822572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5306a0-fa02-427e-8888-9ec4a8546a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:14.824164 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5306a0-fa02-427e-8888-9ec4a8546a21 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:14.835349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1121/2254] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:53:14.839497 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984a4ed-f311-446f-9d9f-49fd331e3b9b admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:14.845557 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:14.851902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984a4ed-f311-446f-9d9f-49fd331e3b9b admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:14.857470 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812ef0b-3cfc-4879-b1a6-672156e9ce16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:14.858153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812ef0b-3cfc-4879-b1a6-672156e9ce16 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:14.858432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812ef0b-3cfc-4879-b1a6-672156e9ce16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:14.858705 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812ef0b-3cfc-4879-b1a6-672156e9ce16 admin admin] PATH_INFO: `/v3/projects/083ae5f8eec64e619ba02d6525c640c5/users/34e64302ff534c5ba985352a801eccf0/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:14.873494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812ef0b-3cfc-4879-b1a6-672156e9ce16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=083ae5f8eec64e619ba02d6525c640c5, user_id=34e64302ff534c5ba985352a801eccf0, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:14.876601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812ef0b-3cfc-4879-b1a6-672156e9ce16 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:14.889880 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1134/2255] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:53:14 2022] PUT /identity/v3/projects/083ae5f8eec64e619ba02d6525c640c5/users/34e64302ff534c5ba985352a801eccf0/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:53:14.901878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9412f-228e-4fb1-a51b-9252c55cfdae None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:14.902339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9412f-228e-4fb1-a51b-9252c55cfdae None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:14.902455 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9412f-228e-4fb1-a51b-9252c55cfdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:14.930863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-fde9412f-228e-4fb1-a51b-9252c55cfdae None None] MFA Rules not processed for user `34e64302ff534c5ba985352a801eccf0`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:14.932096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-fde9412f-228e-4fb1-a51b-9252c55cfdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:14.995733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1122/2256] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:53:15.007752 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812ef0b-3cfc-4879-b1a6-672156e9ce16 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:15.009538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0812ef0b-3cfc-4879-b1a6-672156e9ce16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:15.066709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812ef0b-3cfc-4879-b1a6-672156e9ce16 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:15.070591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e7be3-fc26-4141-8b74-fd2e04d7f588 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:15.071305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e7be3-fc26-4141-8b74-fd2e04d7f588 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:15.071603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e7be3-fc26-4141-8b74-fd2e04d7f588 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:15.071881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e7be3-fc26-4141-8b74-fd2e04d7f588 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:15.081543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e7be3-fc26-4141-8b74-fd2e04d7f588 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:15.083100 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e7be3-fc26-4141-8b74-fd2e04d7f588 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:15.092161 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1135/2257] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:53:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:53:22.433024 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e31e0-1482-45bd-8d91-da2d5ed2edbb None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:22.433551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e31e0-1482-45bd-8d91-da2d5ed2edbb None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:22.433649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e31e0-1482-45bd-8d91-da2d5ed2edbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:22.453943 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-e08e31e0-1482-45bd-8d91-da2d5ed2edbb None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:22.455176 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e08e31e0-1482-45bd-8d91-da2d5ed2edbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:22.524473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1123/2258] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Aug 3 18:53:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:22.570002 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e7be3-fc26-4141-8b74-fd2e04d7f588 service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:22.571451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f35e7be3-fc26-4141-8b74-fd2e04d7f588 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:22.642377 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e7be3-fc26-4141-8b74-fd2e04d7f588 service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:22.648114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c28632-2a5b-4887-9978-ab60f85995fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:22.648798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c28632-2a5b-4887-9978-ab60f85995fe service swift] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:22.649143 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c28632-2a5b-4887-9978-ab60f85995fe service swift] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:22.649570 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c28632-2a5b-4887-9978-ab60f85995fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:22.658423 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c28632-2a5b-4887-9978-ab60f85995fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:22.660019 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c28632-2a5b-4887-9978-ab60f85995fe service swift] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:22.668064 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1136/2259] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 18:53:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 18:53:24.058312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e31e0-1482-45bd-8d91-da2d5ed2edbb None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:24.141571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:24.148828 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e31e0-1482-45bd-8d91-da2d5ed2edbb None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:24.169151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d6a0a-91b1-4a2e-9026-8d5190059990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:24.169898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d6a0a-91b1-4a2e-9026-8d5190059990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:24.171492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d6a0a-91b1-4a2e-9026-8d5190059990 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:24.171979 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d6a0a-91b1-4a2e-9026-8d5190059990 admin admin] PATH_INFO: `/v3/users/34e64302ff534c5ba985352a801eccf0` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:24.173497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d6a0a-91b1-4a2e-9026-8d5190059990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e64302ff534c5ba985352a801eccf0)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:24.175911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d6a0a-91b1-4a2e-9026-8d5190059990 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:24.241168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-418d6a0a-91b1-4a2e-9026-8d5190059990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e64302ff534c5ba985352a801eccf0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:24.250733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-418d6a0a-91b1-4a2e-9026-8d5190059990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e64302ff534c5ba985352a801eccf0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:24.265938 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-418d6a0a-91b1-4a2e-9026-8d5190059990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e64302ff534c5ba985352a801eccf0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:24.274880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-418d6a0a-91b1-4a2e-9026-8d5190059990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e64302ff534c5ba985352a801eccf0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:24.275892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-418d6a0a-91b1-4a2e-9026-8d5190059990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e64302ff534c5ba985352a801eccf0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:24.284569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1124/2260] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:53:24 2022] DELETE /identity/v3/users/34e64302ff534c5ba985352a801eccf0 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:24.290180 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fc6ef-ef31-4b79-a84c-c5180bb00891 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:24.290848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fc6ef-ef31-4b79-a84c-c5180bb00891 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:24.291037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fc6ef-ef31-4b79-a84c-c5180bb00891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:24.404122 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-617fc6ef-ef31-4b79-a84c-c5180bb00891 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:24.411253 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-617fc6ef-ef31-4b79-a84c-c5180bb00891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:24.532326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1137/2261] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:24.544978 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d6a0a-91b1-4a2e-9026-8d5190059990 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:24.547452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-418d6a0a-91b1-4a2e-9026-8d5190059990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:24.676943 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d6a0a-91b1-4a2e-9026-8d5190059990 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:24.681398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9083bd9-282f-4688-85d8-45fc40021d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:24.682045 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9083bd9-282f-4688-85d8-45fc40021d5f service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:24.682329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9083bd9-282f-4688-85d8-45fc40021d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:24.682624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9083bd9-282f-4688-85d8-45fc40021d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:24.694236 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9083bd9-282f-4688-85d8-45fc40021d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:24.695704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9083bd9-282f-4688-85d8-45fc40021d5f service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:24.718608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1125/2262] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:53:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:53:25.825054 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fc6ef-ef31-4b79-a84c-c5180bb00891 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:25.826885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-617fc6ef-ef31-4b79-a84c-c5180bb00891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:25.844275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fc6ef-ef31-4b79-a84c-c5180bb00891 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:25.856909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f81c68-e885-4e4a-be69-1baf36f07fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:25.858063 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f81c68-e885-4e4a-be69-1baf36f07fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:25.858364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f81c68-e885-4e4a-be69-1baf36f07fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:25.858649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f81c68-e885-4e4a-be69-1baf36f07fc7 admin admin] PATH_INFO: `/v3/projects/083ae5f8eec64e619ba02d6525c640c5` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:25.860163 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f81c68-e885-4e4a-be69-1baf36f07fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=083ae5f8eec64e619ba02d6525c640c5)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:25.863602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f81c68-e885-4e4a-be69-1baf36f07fc7 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:25.871048 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:25.939055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-65f81c68-e885-4e4a-be69-1baf36f07fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '083ae5f8eec64e619ba02d6525c640c5', 'request_id': 'req-65f81c68-e885-4e4a-be69-1baf36f07fc7'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:25.956104 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-65f81c68-e885-4e4a-be69-1baf36f07fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '083ae5f8eec64e619ba02d6525c640c5', 'request_id': 'req-65f81c68-e885-4e4a-be69-1baf36f07fc7'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:25.979494 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-65f81c68-e885-4e4a-be69-1baf36f07fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '083ae5f8eec64e619ba02d6525c640c5', 'request_id': 'req-65f81c68-e885-4e4a-be69-1baf36f07fc7'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:25.997506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-65f81c68-e885-4e4a-be69-1baf36f07fc7 admin admin] The token cache is being invalidate because project 083ae5f8eec64e619ba02d6525c640c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:53:25.998133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-65f81c68-e885-4e4a-be69-1baf36f07fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:25.998964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-65f81c68-e885-4e4a-be69-1baf36f07fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:26.000973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1138/2263] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:53:25 2022] DELETE /identity/v3/projects/083ae5f8eec64e619ba02d6525c640c5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:26.007205 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d290d9-4751-4ded-ac2e-aa8f7364b978 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:26.007791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d290d9-4751-4ded-ac2e-aa8f7364b978 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:26.007791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d290d9-4751-4ded-ac2e-aa8f7364b978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:26.029543 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-d5d290d9-4751-4ded-ac2e-aa8f7364b978 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:26.030770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d5d290d9-4751-4ded-ac2e-aa8f7364b978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:26.116986 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1126/2264] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:26.124583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7e95e-6f83-4540-bc15-14246654f0c0 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:26.125093 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7e95e-6f83-4540-bc15-14246654f0c0 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:26.125536 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7e95e-6f83-4540-bc15-14246654f0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:26.152876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-54d7e95e-6f83-4540-bc15-14246654f0c0 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:26.154384 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-54d7e95e-6f83-4540-bc15-14246654f0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:26.180481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1139/2265] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:26.181588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d290d9-4751-4ded-ac2e-aa8f7364b978 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:26.194221 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d290d9-4751-4ded-ac2e-aa8f7364b978 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:26.199469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436f6b9-7bb5-4c0f-bd19-c665f43325b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:26.200110 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436f6b9-7bb5-4c0f-bd19-c665f43325b1 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:26.200424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436f6b9-7bb5-4c0f-bd19-c665f43325b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:26.200705 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436f6b9-7bb5-4c0f-bd19-c665f43325b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:26.201409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436f6b9-7bb5-4c0f-bd19-c665f43325b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:53:26.201713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436f6b9-7bb5-4c0f-bd19-c665f43325b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:26.203306 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436f6b9-7bb5-4c0f-bd19-c665f43325b1 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:26.220320 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1127/2266] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:53:26.223050 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7e95e-6f83-4540-bc15-14246654f0c0 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:26.232978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7e95e-6f83-4540-bc15-14246654f0c0 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:26.237809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568a84d-645b-443e-a574-5c1a2dc21f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:26.238430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568a84d-645b-443e-a574-5c1a2dc21f30 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:26.238712 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568a84d-645b-443e-a574-5c1a2dc21f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:26.238987 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568a84d-645b-443e-a574-5c1a2dc21f30 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:26.239669 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568a84d-645b-443e-a574-5c1a2dc21f30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:26.241742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568a84d-645b-443e-a574-5c1a2dc21f30 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:26.275207 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1140/2267] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:53:26 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:53:26.276398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436f6b9-7bb5-4c0f-bd19-c665f43325b1 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:26.283064 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:26.288715 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436f6b9-7bb5-4c0f-bd19-c665f43325b1 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:26.356169 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa991e-72e1-4907-bcb7-320da9acbee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:26.356832 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa991e-72e1-4907-bcb7-320da9acbee8 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:26.357116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa991e-72e1-4907-bcb7-320da9acbee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:26.357395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa991e-72e1-4907-bcb7-320da9acbee8 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:26.358063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa991e-72e1-4907-bcb7-320da9acbee8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:26.359974 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa991e-72e1-4907-bcb7-320da9acbee8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:26.383897 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1128/2268] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:53:26 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:53:26.387865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568a84d-645b-443e-a574-5c1a2dc21f30 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:26.400953 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568a84d-645b-443e-a574-5c1a2dc21f30 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:26.410373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431545b0-3014-4973-84f7-3aad7decf260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:26.411019 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431545b0-3014-4973-84f7-3aad7decf260 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:26.411309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431545b0-3014-4973-84f7-3aad7decf260 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:26.411650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431545b0-3014-4973-84f7-3aad7decf260 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:26.412348 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431545b0-3014-4973-84f7-3aad7decf260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:26.414083 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431545b0-3014-4973-84f7-3aad7decf260 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:26.428631 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1141/2269] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:53:26.429827 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa991e-72e1-4907-bcb7-320da9acbee8 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:26.441820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa991e-72e1-4907-bcb7-320da9acbee8 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:26.446536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63fc18-ec72-4d97-9cfb-3b5299764895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:26.447149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63fc18-ec72-4d97-9cfb-3b5299764895 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:26.447427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63fc18-ec72-4d97-9cfb-3b5299764895 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:26.447693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63fc18-ec72-4d97-9cfb-3b5299764895 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:26.448424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63fc18-ec72-4d97-9cfb-3b5299764895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:26.450014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63fc18-ec72-4d97-9cfb-3b5299764895 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:26.464866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1129/2270] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:53:26.470467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431545b0-3014-4973-84f7-3aad7decf260 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:26.481291 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431545b0-3014-4973-84f7-3aad7decf260 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:26.493291 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1dfe0-22d7-45da-9317-5e9b5f0ad0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:26.495318 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1dfe0-22d7-45da-9317-5e9b5f0ad0f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:26.495741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1dfe0-22d7-45da-9317-5e9b5f0ad0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:26.496364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1dfe0-22d7-45da-9317-5e9b5f0ad0f3 admin admin] PATH_INFO: `/v3/projects/3718f3d951db43e09941293d168b254d/users/1aee1fd4881d47f892605a22b03b0de8/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:26.529085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1dfe0-22d7-45da-9317-5e9b5f0ad0f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3718f3d951db43e09941293d168b254d, user_id=1aee1fd4881d47f892605a22b03b0de8, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:26.536368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1dfe0-22d7-45da-9317-5e9b5f0ad0f3 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:26.570727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1142/2271] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:53:26 2022] PUT /identity/v3/projects/3718f3d951db43e09941293d168b254d/users/1aee1fd4881d47f892605a22b03b0de8/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:53:26.573405 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e21936-adc9-4654-aba5-9734df0b36eb None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:26.573816 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e21936-adc9-4654-aba5-9734df0b36eb None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:26.574221 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e21936-adc9-4654-aba5-9734df0b36eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:26.606211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1dfe0-22d7-45da-9317-5e9b5f0ad0f3 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:26.610090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e1c1dfe0-22d7-45da-9317-5e9b5f0ad0f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:26.635116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-c5e21936-adc9-4654-aba5-9734df0b36eb None None] MFA Rules not processed for user `1aee1fd4881d47f892605a22b03b0de8`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:26.635116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c5e21936-adc9-4654-aba5-9734df0b36eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:26.816365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1130/2272] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:26.822077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1dfe0-22d7-45da-9317-5e9b5f0ad0f3 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:26.824010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e21936-adc9-4654-aba5-9734df0b36eb None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:26.826749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c5e21936-adc9-4654-aba5-9734df0b36eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:26.837760 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def5bfd-dde4-4c96-9df0-6aa6c22871a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:26.838435 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def5bfd-dde4-4c96-9df0-6aa6c22871a1 service placement] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:26.838723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def5bfd-dde4-4c96-9df0-6aa6c22871a1 service placement] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:26.838999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def5bfd-dde4-4c96-9df0-6aa6c22871a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:26.842190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7def5bfd-dde4-4c96-9df0-6aa6c22871a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:26.873352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def5bfd-dde4-4c96-9df0-6aa6c22871a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:26.876804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def5bfd-dde4-4c96-9df0-6aa6c22871a1 service placement] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:26.896775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1143/2273] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:53:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:53:26.929918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e21936-adc9-4654-aba5-9734df0b36eb None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:26.934837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ec6ad-a225-4a81-b6a5-ee0030ba445d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:26.936162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ec6ad-a225-4a81-b6a5-ee0030ba445d service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:26.936839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ec6ad-a225-4a81-b6a5-ee0030ba445d service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:26.937222 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ec6ad-a225-4a81-b6a5-ee0030ba445d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:26.947932 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ec6ad-a225-4a81-b6a5-ee0030ba445d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:26.950994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ec6ad-a225-4a81-b6a5-ee0030ba445d service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:26.963009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1131/2274] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:53:26 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:53:36.886504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:36.957411 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:44.503532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f71ef-d5f9-4b96-a778-a4674c282217 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:44.504032 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f71ef-d5f9-4b96-a778-a4674c282217 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:44.504137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f71ef-d5f9-4b96-a778-a4674c282217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:44.523434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-599f71ef-d5f9-4b96-a778-a4674c282217 None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:44.524704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-599f71ef-d5f9-4b96-a778-a4674c282217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:44.580347 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1144/2275] 10.222.0.69 () {58 vars in 967 bytes} [Wed Aug 3 18:53:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:44.593788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ec6ad-a225-4a81-b6a5-ee0030ba445d service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:44.595314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-612ec6ad-a225-4a81-b6a5-ee0030ba445d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:44.670034 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ec6ad-a225-4a81-b6a5-ee0030ba445d service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:44.675129 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff84f2-f8f0-4ba3-a39b-471e4a332052 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:44.675681 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff84f2-f8f0-4ba3-a39b-471e4a332052 service swift] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:44.675966 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff84f2-f8f0-4ba3-a39b-471e4a332052 service swift] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:44.676244 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff84f2-f8f0-4ba3-a39b-471e4a332052 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:44.684364 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff84f2-f8f0-4ba3-a39b-471e4a332052 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:44.685964 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff84f2-f8f0-4ba3-a39b-471e4a332052 service swift] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:44.694172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1132/2276] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 18:53:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 18:53:51.003545 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7fc4c-da0c-4609-b0e2-5b2ed3a1078c None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:51.004677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7fc4c-da0c-4609-b0e2-5b2ed3a1078c None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:51.004677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7fc4c-da0c-4609-b0e2-5b2ed3a1078c None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:51.005663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1145/2277] 10.222.0.69 () {54 vars in 921 bytes} [Wed Aug 3 18:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:53:51.042548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae5303-7628-41e0-96ae-530c652c98d0 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:51.043087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae5303-7628-41e0-96ae-530c652c98d0 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:51.043087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae5303-7628-41e0-96ae-530c652c98d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:51.064732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-93ae5303-7628-41e0-96ae-530c652c98d0 None None] MFA Rules not processed for user `fea93ceb4cbc49c2b4b280a4ddc257b7`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:51.065945 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-93ae5303-7628-41e0-96ae-530c652c98d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:51.071553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7fc4c-da0c-4609-b0e2-5b2ed3a1078c None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:51.073270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-22d7fc4c-da0c-4609-b0e2-5b2ed3a1078c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:51.075707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1133/2278] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Aug 3 18:53:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:51.099601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae5303-7628-41e0-96ae-530c652c98d0 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:51.102637 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-93ae5303-7628-41e0-96ae-530c652c98d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:51.116624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae5303-7628-41e0-96ae-530c652c98d0 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:51.121776 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0002f1-3462-46a4-92f2-0ad4d954c211 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:51.122393 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0002f1-3462-46a4-92f2-0ad4d954c211 service placement] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:51.122676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0002f1-3462-46a4-92f2-0ad4d954c211 service placement] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:51.122950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0002f1-3462-46a4-92f2-0ad4d954c211 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:51.132339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0002f1-3462-46a4-92f2-0ad4d954c211 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:51.133919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0002f1-3462-46a4-92f2-0ad4d954c211 service placement] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:51.145664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1134/2279] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:53:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:53:51.151577 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7fc4c-da0c-4609-b0e2-5b2ed3a1078c None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:51.156470 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dbbc0-1c50-4fea-9448-48a5f386f9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:51.157119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dbbc0-1c50-4fea-9448-48a5f386f9dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:51.157408 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dbbc0-1c50-4fea-9448-48a5f386f9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:51.157691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dbbc0-1c50-4fea-9448-48a5f386f9dc admin admin] PATH_INFO: `/v3/users/9201a1d107bf41959751e7d7c4e4d37d` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:51.158894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76dbbc0-1c50-4fea-9448-48a5f386f9dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9201a1d107bf41959751e7d7c4e4d37d)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:51.161467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76dbbc0-1c50-4fea-9448-48a5f386f9dc admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:51.209200 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e76dbbc0-1c50-4fea-9448-48a5f386f9dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9201a1d107bf41959751e7d7c4e4d37d'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:51.216441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e76dbbc0-1c50-4fea-9448-48a5f386f9dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9201a1d107bf41959751e7d7c4e4d37d'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:51.228659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e76dbbc0-1c50-4fea-9448-48a5f386f9dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9201a1d107bf41959751e7d7c4e4d37d'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:51.235546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e76dbbc0-1c50-4fea-9448-48a5f386f9dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9201a1d107bf41959751e7d7c4e4d37d'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:51.236221 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e76dbbc0-1c50-4fea-9448-48a5f386f9dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9201a1d107bf41959751e7d7c4e4d37d'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:51.242799 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1146/2280] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:53:51 2022] DELETE /identity/v3/users/9201a1d107bf41959751e7d7c4e4d37d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:51.421972 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0002f1-3462-46a4-92f2-0ad4d954c211 service placement] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:51.434407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0002f1-3462-46a4-92f2-0ad4d954c211 service placement] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:51.492614 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a970273-7ebd-4708-ac44-481bfeca3d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:51.493257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a970273-7ebd-4708-ac44-481bfeca3d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:51.493535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a970273-7ebd-4708-ac44-481bfeca3d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:51.494048 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a970273-7ebd-4708-ac44-481bfeca3d62 admin admin] PATH_INFO: `/v3/projects/1fe2ae076eda4afa99c6c28464e1377b` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:51.495205 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a970273-7ebd-4708-ac44-481bfeca3d62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fe2ae076eda4afa99c6c28464e1377b)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:51.497263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a970273-7ebd-4708-ac44-481bfeca3d62 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:51.544235 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3a970273-7ebd-4708-ac44-481bfeca3d62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fe2ae076eda4afa99c6c28464e1377b', 'request_id': 'req-3a970273-7ebd-4708-ac44-481bfeca3d62'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:51.549810 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3a970273-7ebd-4708-ac44-481bfeca3d62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fe2ae076eda4afa99c6c28464e1377b', 'request_id': 'req-3a970273-7ebd-4708-ac44-481bfeca3d62'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:51.556655 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3a970273-7ebd-4708-ac44-481bfeca3d62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fe2ae076eda4afa99c6c28464e1377b', 'request_id': 'req-3a970273-7ebd-4708-ac44-481bfeca3d62'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:51.567413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:51.572105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3a970273-7ebd-4708-ac44-481bfeca3d62 admin admin] The token cache is being invalidate because project 1fe2ae076eda4afa99c6c28464e1377b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:53:51.572910 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3a970273-7ebd-4708-ac44-481bfeca3d62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:51.573863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-3a970273-7ebd-4708-ac44-481bfeca3d62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:51.576006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1135/2281] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:53:51 2022] DELETE /identity/v3/projects/1fe2ae076eda4afa99c6c28464e1377b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:51.580512 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceacc37-3137-4e6e-9d2b-71344bf54096 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:51.581138 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceacc37-3137-4e6e-9d2b-71344bf54096 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:51.581675 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceacc37-3137-4e6e-9d2b-71344bf54096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:51.605256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-aceacc37-3137-4e6e-9d2b-71344bf54096 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:51.606492 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-aceacc37-3137-4e6e-9d2b-71344bf54096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:51.628096 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:51.674554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1147/2282] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:53:51.683546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520063c-1f31-41f1-b324-f9226fea37d2 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:51.683837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520063c-1f31-41f1-b324-f9226fea37d2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:51.684136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520063c-1f31-41f1-b324-f9226fea37d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:51.704645 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-8520063c-1f31-41f1-b324-f9226fea37d2 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:51.705868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8520063c-1f31-41f1-b324-f9226fea37d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:51.719229 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1136/2283] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:51.723842 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceacc37-3137-4e6e-9d2b-71344bf54096 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:51.734072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceacc37-3137-4e6e-9d2b-71344bf54096 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:51.739168 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf7832-4289-40ec-a715-26709bc0d145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:51.739790 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf7832-4289-40ec-a715-26709bc0d145 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:51.740198 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf7832-4289-40ec-a715-26709bc0d145 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:51.740554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf7832-4289-40ec-a715-26709bc0d145 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:51.741268 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf7832-4289-40ec-a715-26709bc0d145 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:53:51.741574 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf7832-4289-40ec-a715-26709bc0d145 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:51.743160 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf7832-4289-40ec-a715-26709bc0d145 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:51.755126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1148/2284] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:53:51.759453 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520063c-1f31-41f1-b324-f9226fea37d2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:51.769117 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520063c-1f31-41f1-b324-f9226fea37d2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:51.774198 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a35c9a-149b-4bcc-b41a-9ef1026d9390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:51.774800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a35c9a-149b-4bcc-b41a-9ef1026d9390 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:51.775080 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a35c9a-149b-4bcc-b41a-9ef1026d9390 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:51.775350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a35c9a-149b-4bcc-b41a-9ef1026d9390 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:51.776012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a35c9a-149b-4bcc-b41a-9ef1026d9390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:51.777954 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a35c9a-149b-4bcc-b41a-9ef1026d9390 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:51.805509 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1137/2285] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:53:51 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:53:51.809576 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf7832-4289-40ec-a715-26709bc0d145 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:51.819471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf7832-4289-40ec-a715-26709bc0d145 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:51.875879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb37c2-87c5-4e24-bee5-9ec27ed12251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:51.876452 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb37c2-87c5-4e24-bee5-9ec27ed12251 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:51.876738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb37c2-87c5-4e24-bee5-9ec27ed12251 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:51.877026 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb37c2-87c5-4e24-bee5-9ec27ed12251 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:51.877671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb37c2-87c5-4e24-bee5-9ec27ed12251 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:51.879877 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb37c2-87c5-4e24-bee5-9ec27ed12251 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:51.902444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1149/2286] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:53:51 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:53:51.906710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a35c9a-149b-4bcc-b41a-9ef1026d9390 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:51.921017 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a35c9a-149b-4bcc-b41a-9ef1026d9390 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:51.926100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507db6a2-a407-4d1e-86b7-a81b4f98d8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:51.926734 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507db6a2-a407-4d1e-86b7-a81b4f98d8ef admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:51.927018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507db6a2-a407-4d1e-86b7-a81b4f98d8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:51.927329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507db6a2-a407-4d1e-86b7-a81b4f98d8ef admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:51.928062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507db6a2-a407-4d1e-86b7-a81b4f98d8ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:51.929892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507db6a2-a407-4d1e-86b7-a81b4f98d8ef admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:51.942696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1138/2287] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:53:51.946891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb37c2-87c5-4e24-bee5-9ec27ed12251 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:51.960832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb37c2-87c5-4e24-bee5-9ec27ed12251 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:51.967791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f59d8a-bfc8-4e21-8415-4a216a80e315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:51.968455 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f59d8a-bfc8-4e21-8415-4a216a80e315 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:51.968731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f59d8a-bfc8-4e21-8415-4a216a80e315 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:51.969003 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f59d8a-bfc8-4e21-8415-4a216a80e315 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:51.969699 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f59d8a-bfc8-4e21-8415-4a216a80e315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:51.971342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f59d8a-bfc8-4e21-8415-4a216a80e315 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:51.983831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1150/2288] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:53:51.987905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507db6a2-a407-4d1e-86b7-a81b4f98d8ef admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:52.004980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507db6a2-a407-4d1e-86b7-a81b4f98d8ef admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:52.009662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe21c19-bec2-42e8-9e73-dc696f2d59c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:52.010330 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe21c19-bec2-42e8-9e73-dc696f2d59c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:52.010614 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe21c19-bec2-42e8-9e73-dc696f2d59c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:52.010891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe21c19-bec2-42e8-9e73-dc696f2d59c8 admin admin] PATH_INFO: `/v3/projects/b440acf4b71c46cc81602cd40c80ff8a/users/ef11980490564c11aaa14404190c764d/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:52.031832 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe21c19-bec2-42e8-9e73-dc696f2d59c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b440acf4b71c46cc81602cd40c80ff8a, user_id=ef11980490564c11aaa14404190c764d, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:52.034531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe21c19-bec2-42e8-9e73-dc696f2d59c8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:52.049085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1139/2289] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:53:51 2022] PUT /identity/v3/projects/b440acf4b71c46cc81602cd40c80ff8a/users/ef11980490564c11aaa14404190c764d/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:52.116321 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09471a-a5c9-44bf-86f2-382656e849d8 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:52.116862 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09471a-a5c9-44bf-86f2-382656e849d8 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:52.116862 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09471a-a5c9-44bf-86f2-382656e849d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:52.146173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-4f09471a-a5c9-44bf-86f2-382656e849d8 None None] MFA Rules not processed for user `ef11980490564c11aaa14404190c764d`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:52.147361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4f09471a-a5c9-44bf-86f2-382656e849d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:52.214020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1151/2290] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:52.225278 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe21c19-bec2-42e8-9e73-dc696f2d59c8 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:52.227032 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9fe21c19-bec2-42e8-9e73-dc696f2d59c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:52.283752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe21c19-bec2-42e8-9e73-dc696f2d59c8 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:52.288179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a1554-93f4-4c90-a9bc-5a9d86523d12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:52.288827 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a1554-93f4-4c90-a9bc-5a9d86523d12 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:52.289109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a1554-93f4-4c90-a9bc-5a9d86523d12 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:52.289385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a1554-93f4-4c90-a9bc-5a9d86523d12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:52.298068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a1554-93f4-4c90-a9bc-5a9d86523d12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:52.299495 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a1554-93f4-4c90-a9bc-5a9d86523d12 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:52.309140 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1140/2291] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:53:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:53:54.939555 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09471a-a5c9-44bf-86f2-382656e849d8 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:54.941741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4f09471a-a5c9-44bf-86f2-382656e849d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:54.964349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:55.018453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09471a-a5c9-44bf-86f2-382656e849d8 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:55.024791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26829f87-f21a-4386-8c67-5fbce73fa92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:55.025474 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26829f87-f21a-4386-8c67-5fbce73fa92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:55.025759 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26829f87-f21a-4386-8c67-5fbce73fa92e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:55.026038 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26829f87-f21a-4386-8c67-5fbce73fa92e admin admin] PATH_INFO: `/v3/users/1aee1fd4881d47f892605a22b03b0de8` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:55.027244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26829f87-f21a-4386-8c67-5fbce73fa92e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aee1fd4881d47f892605a22b03b0de8)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:55.029553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26829f87-f21a-4386-8c67-5fbce73fa92e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:55.093020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-26829f87-f21a-4386-8c67-5fbce73fa92e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aee1fd4881d47f892605a22b03b0de8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:55.105775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-26829f87-f21a-4386-8c67-5fbce73fa92e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aee1fd4881d47f892605a22b03b0de8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:55.125430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-26829f87-f21a-4386-8c67-5fbce73fa92e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aee1fd4881d47f892605a22b03b0de8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:55.135127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-26829f87-f21a-4386-8c67-5fbce73fa92e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aee1fd4881d47f892605a22b03b0de8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:55.135862 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-26829f87-f21a-4386-8c67-5fbce73fa92e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aee1fd4881d47f892605a22b03b0de8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:55.143423 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1152/2292] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:53:54 2022] DELETE /identity/v3/users/1aee1fd4881d47f892605a22b03b0de8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:55.148501 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fde17-a098-47a3-add9-7a86c4c1e0a1 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:55.149118 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fde17-a098-47a3-add9-7a86c4c1e0a1 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:55.149118 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fde17-a098-47a3-add9-7a86c4c1e0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:55.168418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-202fde17-a098-47a3-add9-7a86c4c1e0a1 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:55.169664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-202fde17-a098-47a3-add9-7a86c4c1e0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:55.227863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1141/2293] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:55.237399 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26829f87-f21a-4386-8c67-5fbce73fa92e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:55.239923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-26829f87-f21a-4386-8c67-5fbce73fa92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:55.298124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26829f87-f21a-4386-8c67-5fbce73fa92e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:55.302722 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebcef3-dad9-496b-bc99-e07f525a6af4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:55.303350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebcef3-dad9-496b-bc99-e07f525a6af4 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:55.303628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebcef3-dad9-496b-bc99-e07f525a6af4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:55.303898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebcef3-dad9-496b-bc99-e07f525a6af4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:55.313261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebcef3-dad9-496b-bc99-e07f525a6af4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:55.314691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebcef3-dad9-496b-bc99-e07f525a6af4 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:55.324792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1153/2294] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:53:55.597975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fde17-a098-47a3-add9-7a86c4c1e0a1 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:55.600263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-202fde17-a098-47a3-add9-7a86c4c1e0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:55.614082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fde17-a098-47a3-add9-7a86c4c1e0a1 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:55.619254 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b618c0e-c8e1-475d-b401-23f55ecfcc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:55.619911 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b618c0e-c8e1-475d-b401-23f55ecfcc31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:55.620193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b618c0e-c8e1-475d-b401-23f55ecfcc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:55.620523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b618c0e-c8e1-475d-b401-23f55ecfcc31 admin admin] PATH_INFO: `/v3/projects/3718f3d951db43e09941293d168b254d` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:55.621708 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b618c0e-c8e1-475d-b401-23f55ecfcc31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3718f3d951db43e09941293d168b254d)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:55.623672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b618c0e-c8e1-475d-b401-23f55ecfcc31 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:55.661906 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:55.674624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7b618c0e-c8e1-475d-b401-23f55ecfcc31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3718f3d951db43e09941293d168b254d', 'request_id': 'req-7b618c0e-c8e1-475d-b401-23f55ecfcc31'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:55.681261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7b618c0e-c8e1-475d-b401-23f55ecfcc31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3718f3d951db43e09941293d168b254d', 'request_id': 'req-7b618c0e-c8e1-475d-b401-23f55ecfcc31'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:55.687487 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7b618c0e-c8e1-475d-b401-23f55ecfcc31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3718f3d951db43e09941293d168b254d', 'request_id': 'req-7b618c0e-c8e1-475d-b401-23f55ecfcc31'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:55.701239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7b618c0e-c8e1-475d-b401-23f55ecfcc31 admin admin] The token cache is being invalidate because project 3718f3d951db43e09941293d168b254d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:53:55.701526 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7b618c0e-c8e1-475d-b401-23f55ecfcc31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:55.702149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7b618c0e-c8e1-475d-b401-23f55ecfcc31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:53:55.703704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1142/2295] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:53:55 2022] DELETE /identity/v3/projects/3718f3d951db43e09941293d168b254d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:53:55.709550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8cbbe-fe29-4ffb-8296-0261a80cb7aa None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:55.709841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8cbbe-fe29-4ffb-8296-0261a80cb7aa None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:55.710117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8cbbe-fe29-4ffb-8296-0261a80cb7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:55.729966 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-daa8cbbe-fe29-4ffb-8296-0261a80cb7aa None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:55.731549 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-daa8cbbe-fe29-4ffb-8296-0261a80cb7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:55.772676 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:53:55.796499 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1154/2296] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:55.805632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddce09d-e317-44df-9928-40e50fbda798 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:55.805919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddce09d-e317-44df-9928-40e50fbda798 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:55.806196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddce09d-e317-44df-9928-40e50fbda798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:55.826218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-fddce09d-e317-44df-9928-40e50fbda798 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:55.827431 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-fddce09d-e317-44df-9928-40e50fbda798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:55.836473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1143/2297] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:53:55.840788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8cbbe-fe29-4ffb-8296-0261a80cb7aa None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:55.850618 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8cbbe-fe29-4ffb-8296-0261a80cb7aa None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:55.855221 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8b2f9-e365-4aa6-a5ba-6aa2a45ced9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:55.855814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8b2f9-e365-4aa6-a5ba-6aa2a45ced9f admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:55.856097 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8b2f9-e365-4aa6-a5ba-6aa2a45ced9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:55.856402 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8b2f9-e365-4aa6-a5ba-6aa2a45ced9f admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:55.857105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8b2f9-e365-4aa6-a5ba-6aa2a45ced9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:53:55.857442 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8b2f9-e365-4aa6-a5ba-6aa2a45ced9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:55.859035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8b2f9-e365-4aa6-a5ba-6aa2a45ced9f admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:55.871292 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1155/2298] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:53:55.875385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddce09d-e317-44df-9928-40e50fbda798 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:55.885531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddce09d-e317-44df-9928-40e50fbda798 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:55.890472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887c687-d369-45c1-9ba8-89dd5cd2ebe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:55.891116 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887c687-d369-45c1-9ba8-89dd5cd2ebe0 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:55.891401 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887c687-d369-45c1-9ba8-89dd5cd2ebe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:55.891680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887c687-d369-45c1-9ba8-89dd5cd2ebe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:55.892391 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887c687-d369-45c1-9ba8-89dd5cd2ebe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:55.894337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887c687-d369-45c1-9ba8-89dd5cd2ebe0 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:55.921625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1144/2299] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:53:55 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:53:55.926850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8b2f9-e365-4aa6-a5ba-6aa2a45ced9f admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:55.937566 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8b2f9-e365-4aa6-a5ba-6aa2a45ced9f admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:56.000350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddbf257-880f-41fe-9310-d7f56375585e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:56.001840 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbf257-880f-41fe-9310-d7f56375585e admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:56.002140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbf257-880f-41fe-9310-d7f56375585e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:56.002576 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbf257-880f-41fe-9310-d7f56375585e admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:56.003514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddbf257-880f-41fe-9310-d7f56375585e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:56.006147 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddbf257-880f-41fe-9310-d7f56375585e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:56.030325 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1156/2300] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:53:55 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:53:56.034739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887c687-d369-45c1-9ba8-89dd5cd2ebe0 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:56.047458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887c687-d369-45c1-9ba8-89dd5cd2ebe0 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:56.053411 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12776ee-513d-4512-9313-1611eeff0adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:56.054037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12776ee-513d-4512-9313-1611eeff0adf admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:56.054318 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12776ee-513d-4512-9313-1611eeff0adf admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:56.054598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12776ee-513d-4512-9313-1611eeff0adf admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:56.055298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12776ee-513d-4512-9313-1611eeff0adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:56.056993 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12776ee-513d-4512-9313-1611eeff0adf admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:56.068066 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1145/2301] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:53:56.072260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddbf257-880f-41fe-9310-d7f56375585e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:56.082612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddbf257-880f-41fe-9310-d7f56375585e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:56.087381 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b40c8-0d3d-4484-b8e1-98754ac63840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:56.087990 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b40c8-0d3d-4484-b8e1-98754ac63840 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:56.088274 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b40c8-0d3d-4484-b8e1-98754ac63840 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:56.088578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b40c8-0d3d-4484-b8e1-98754ac63840 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:56.089302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b40c8-0d3d-4484-b8e1-98754ac63840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:56.090910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b40c8-0d3d-4484-b8e1-98754ac63840 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:56.101670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1157/2302] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:53:56.105737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12776ee-513d-4512-9313-1611eeff0adf admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:56.116195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12776ee-513d-4512-9313-1611eeff0adf admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:56.120828 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d663e2d-e1f9-44b7-bd80-d19ab72c8ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:56.121929 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d663e2d-e1f9-44b7-bd80-d19ab72c8ab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:56.121929 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d663e2d-e1f9-44b7-bd80-d19ab72c8ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:56.122207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d663e2d-e1f9-44b7-bd80-d19ab72c8ab8 admin admin] PATH_INFO: `/v3/projects/ae44271cd18840139878627231952299/users/d8893baadb774d4aaace19e8fa12ae82/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:56.138022 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d663e2d-e1f9-44b7-bd80-d19ab72c8ab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae44271cd18840139878627231952299, user_id=d8893baadb774d4aaace19e8fa12ae82, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:56.140825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d663e2d-e1f9-44b7-bd80-d19ab72c8ab8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:56.154393 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1146/2303] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:53:56 2022] PUT /identity/v3/projects/ae44271cd18840139878627231952299/users/d8893baadb774d4aaace19e8fa12ae82/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:53:56.166148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e0bfc-51f7-4a68-ad76-2503836161db None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:56.166574 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e0bfc-51f7-4a68-ad76-2503836161db None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:56.166856 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e0bfc-51f7-4a68-ad76-2503836161db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:56.196885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d82e0bfc-51f7-4a68-ad76-2503836161db None None] MFA Rules not processed for user `d8893baadb774d4aaace19e8fa12ae82`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:53:56.198239 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d82e0bfc-51f7-4a68-ad76-2503836161db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:56.263437 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1158/2304] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:53:56 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:53:56.274849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d663e2d-e1f9-44b7-bd80-d19ab72c8ab8 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:53:56.276583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-5d663e2d-e1f9-44b7-bd80-d19ab72c8ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:53:56.334886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d663e2d-e1f9-44b7-bd80-d19ab72c8ab8 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:53:56.342684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb5dea-eb58-42b9-a5d8-423f2e9166c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:53:56.342684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb5dea-eb58-42b9-a5d8-423f2e9166c6 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:53:56.342684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb5dea-eb58-42b9-a5d8-423f2e9166c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:53:56.342684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb5dea-eb58-42b9-a5d8-423f2e9166c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:53:56.359407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb5dea-eb58-42b9-a5d8-423f2e9166c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:53:56.359407 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb5dea-eb58-42b9-a5d8-423f2e9166c6 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:53:56.374483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1147/2305] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:53:56 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:54:06.260574 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:54:06.364835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:DELETE=5,keystone:SELECT=64,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:54:15.807362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e0bfc-51f7-4a68-ad76-2503836161db None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:15.871333 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e0bfc-51f7-4a68-ad76-2503836161db None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:15.875824 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005e674-1574-4943-b0bb-84130871886e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:15.876495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005e674-1574-4943-b0bb-84130871886e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:15.876772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005e674-1574-4943-b0bb-84130871886e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:15.877044 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005e674-1574-4943-b0bb-84130871886e admin admin] PATH_INFO: `/v3/users/d8893baadb774d4aaace19e8fa12ae82` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:15.878452 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005e674-1574-4943-b0bb-84130871886e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8893baadb774d4aaace19e8fa12ae82)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:15.880404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005e674-1574-4943-b0bb-84130871886e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:15.927571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b005e674-1574-4943-b0bb-84130871886e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8893baadb774d4aaace19e8fa12ae82'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:15.936571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b005e674-1574-4943-b0bb-84130871886e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8893baadb774d4aaace19e8fa12ae82'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:15.949533 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b005e674-1574-4943-b0bb-84130871886e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8893baadb774d4aaace19e8fa12ae82'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:15.956380 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b005e674-1574-4943-b0bb-84130871886e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8893baadb774d4aaace19e8fa12ae82'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:15.957036 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b005e674-1574-4943-b0bb-84130871886e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8893baadb774d4aaace19e8fa12ae82'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:15.963521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1159/2306] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:54:15 2022] DELETE /identity/v3/users/d8893baadb774d4aaace19e8fa12ae82 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:54:15.968044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec40435-d1b0-4f93-a7de-0cbfa74379f4 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:15.968525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec40435-d1b0-4f93-a7de-0cbfa74379f4 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:15.968637 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec40435-d1b0-4f93-a7de-0cbfa74379f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:15.988073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-2ec40435-d1b0-4f93-a7de-0cbfa74379f4 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:54:15.989339 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-2ec40435-d1b0-4f93-a7de-0cbfa74379f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:16.051604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1148/2307] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:54:16.061769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005e674-1574-4943-b0bb-84130871886e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:16.063197 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b005e674-1574-4943-b0bb-84130871886e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:16.126336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005e674-1574-4943-b0bb-84130871886e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:16.130389 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb21a2-67c8-4385-94f2-7359df238a5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:16.131660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb21a2-67c8-4385-94f2-7359df238a5a service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:16.131947 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb21a2-67c8-4385-94f2-7359df238a5a service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:16.132219 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb21a2-67c8-4385-94f2-7359df238a5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:16.142725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb21a2-67c8-4385-94f2-7359df238a5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:16.144163 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb21a2-67c8-4385-94f2-7359df238a5a service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:16.155961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1160/2308] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:54:16.479589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec40435-d1b0-4f93-a7de-0cbfa74379f4 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:16.481176 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-2ec40435-d1b0-4f93-a7de-0cbfa74379f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:16.494849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec40435-d1b0-4f93-a7de-0cbfa74379f4 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:16.498593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deaee56-035c-438a-a198-811bad9c8976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:16.499252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaee56-035c-438a-a198-811bad9c8976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:16.499538 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaee56-035c-438a-a198-811bad9c8976 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:16.499819 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaee56-035c-438a-a198-811bad9c8976 admin admin] PATH_INFO: `/v3/projects/ae44271cd18840139878627231952299` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:16.501133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deaee56-035c-438a-a198-811bad9c8976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae44271cd18840139878627231952299)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:16.503298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deaee56-035c-438a-a198-811bad9c8976 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:16.553817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5deaee56-035c-438a-a198-811bad9c8976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae44271cd18840139878627231952299', 'request_id': 'req-5deaee56-035c-438a-a198-811bad9c8976'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:16.560682 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5deaee56-035c-438a-a198-811bad9c8976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae44271cd18840139878627231952299', 'request_id': 'req-5deaee56-035c-438a-a198-811bad9c8976'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:16.567343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5deaee56-035c-438a-a198-811bad9c8976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae44271cd18840139878627231952299', 'request_id': 'req-5deaee56-035c-438a-a198-811bad9c8976'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:16.581265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5deaee56-035c-438a-a198-811bad9c8976 admin admin] The token cache is being invalidate because project ae44271cd18840139878627231952299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:54:16.581533 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5deaee56-035c-438a-a198-811bad9c8976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:16.582177 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5deaee56-035c-438a-a198-811bad9c8976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:16.583706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1149/2309] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:54:16 2022] DELETE /identity/v3/projects/ae44271cd18840139878627231952299 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:54:16.589211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986c763-85dc-4768-b7b0-2c409f1367fa None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:16.589500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986c763-85dc-4768-b7b0-2c409f1367fa None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:16.589785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986c763-85dc-4768-b7b0-2c409f1367fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:16.610463 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-3986c763-85dc-4768-b7b0-2c409f1367fa None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:54:16.611692 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-3986c763-85dc-4768-b7b0-2c409f1367fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:16.630918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:54:16.680687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1161/2310] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:54:16.690580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c592d-b02f-4935-ab83-af978d1eda79 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:16.690580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c592d-b02f-4935-ab83-af978d1eda79 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:16.690580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c592d-b02f-4935-ab83-af978d1eda79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:16.713947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-799c592d-b02f-4935-ab83-af978d1eda79 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:54:16.715152 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-799c592d-b02f-4935-ab83-af978d1eda79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:16.723791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1150/2311] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:54:16.728441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986c763-85dc-4768-b7b0-2c409f1367fa None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:16.738913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986c763-85dc-4768-b7b0-2c409f1367fa None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:16.745457 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f0c5b-1d60-4fa1-b4d5-4ac80a83ea45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:16.746087 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f0c5b-1d60-4fa1-b4d5-4ac80a83ea45 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:16.746374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f0c5b-1d60-4fa1-b4d5-4ac80a83ea45 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:16.746651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f0c5b-1d60-4fa1-b4d5-4ac80a83ea45 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:16.747438 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f0c5b-1d60-4fa1-b4d5-4ac80a83ea45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:54:16.747751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f0c5b-1d60-4fa1-b4d5-4ac80a83ea45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:16.749462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f0c5b-1d60-4fa1-b4d5-4ac80a83ea45 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:16.768002 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1162/2312] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:54:16.772795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c592d-b02f-4935-ab83-af978d1eda79 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:16.782654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c592d-b02f-4935-ab83-af978d1eda79 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:16.787387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e8c02-411d-4a24-b53a-10ca623a1b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:16.788005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e8c02-411d-4a24-b53a-10ca623a1b12 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:16.788305 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e8c02-411d-4a24-b53a-10ca623a1b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:16.788581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e8c02-411d-4a24-b53a-10ca623a1b12 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:16.789258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e8c02-411d-4a24-b53a-10ca623a1b12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:16.791829 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e8c02-411d-4a24-b53a-10ca623a1b12 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:16.819267 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1151/2313] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:54:16 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:54:16.823740 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f0c5b-1d60-4fa1-b4d5-4ac80a83ea45 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:16.833981 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f0c5b-1d60-4fa1-b4d5-4ac80a83ea45 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:17.016607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2801e-b73e-4b8f-a4dd-d1961e4aff8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:17.017233 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2801e-b73e-4b8f-a4dd-d1961e4aff8c admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:17.017519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2801e-b73e-4b8f-a4dd-d1961e4aff8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:17.017795 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2801e-b73e-4b8f-a4dd-d1961e4aff8c admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:17.018470 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2801e-b73e-4b8f-a4dd-d1961e4aff8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:17.020396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2801e-b73e-4b8f-a4dd-d1961e4aff8c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:17.043220 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1163/2314] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:54:16 2022] POST /identity/v3/users => generated 292 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:54:17.047642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e8c02-411d-4a24-b53a-10ca623a1b12 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:17.058443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e8c02-411d-4a24-b53a-10ca623a1b12 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:17.063058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219a0fc-c17f-4c4f-bb8b-9e69b55ab30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:17.063672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219a0fc-c17f-4c4f-bb8b-9e69b55ab30a admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:17.063955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219a0fc-c17f-4c4f-bb8b-9e69b55ab30a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:17.064252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219a0fc-c17f-4c4f-bb8b-9e69b55ab30a admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:17.064996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219a0fc-c17f-4c4f-bb8b-9e69b55ab30a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:17.066582 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219a0fc-c17f-4c4f-bb8b-9e69b55ab30a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:17.077534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1152/2315] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:54:17.081429 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2801e-b73e-4b8f-a4dd-d1961e4aff8c admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:17.092026 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2801e-b73e-4b8f-a4dd-d1961e4aff8c admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:17.096681 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d1aad-f4f3-48c3-931d-b9d3e8196423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:17.097287 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d1aad-f4f3-48c3-931d-b9d3e8196423 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:17.097572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d1aad-f4f3-48c3-931d-b9d3e8196423 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:17.097848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d1aad-f4f3-48c3-931d-b9d3e8196423 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:17.098543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d1aad-f4f3-48c3-931d-b9d3e8196423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:17.100139 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d1aad-f4f3-48c3-931d-b9d3e8196423 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:17.110708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1164/2316] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:54:17.114834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219a0fc-c17f-4c4f-bb8b-9e69b55ab30a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:17.125489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219a0fc-c17f-4c4f-bb8b-9e69b55ab30a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:17.130314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012432a3-e48b-473f-81e5-552e0d924817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:17.130975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012432a3-e48b-473f-81e5-552e0d924817 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:17.131259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012432a3-e48b-473f-81e5-552e0d924817 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:17.131535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012432a3-e48b-473f-81e5-552e0d924817 admin admin] PATH_INFO: `/v3/projects/182c4b8842bb4aa1a31fbd603c52b276/users/effc38ce5cd6441b92ed3bbb2eaee76c/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:17.146067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012432a3-e48b-473f-81e5-552e0d924817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=182c4b8842bb4aa1a31fbd603c52b276, user_id=effc38ce5cd6441b92ed3bbb2eaee76c, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:17.149239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012432a3-e48b-473f-81e5-552e0d924817 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:17.161923 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1153/2317] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:54:17 2022] PUT /identity/v3/projects/182c4b8842bb4aa1a31fbd603c52b276/users/effc38ce5cd6441b92ed3bbb2eaee76c/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:54:17.167642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abf57e-bae3-47a3-8451-62a50e246e64 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:17.167963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abf57e-bae3-47a3-8451-62a50e246e64 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:17.168237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abf57e-bae3-47a3-8451-62a50e246e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:17.187660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-00abf57e-bae3-47a3-8451-62a50e246e64 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:54:17.189890 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-00abf57e-bae3-47a3-8451-62a50e246e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:17.249386 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1165/2318] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:54:17.259218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012432a3-e48b-473f-81e5-552e0d924817 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:17.260947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-012432a3-e48b-473f-81e5-552e0d924817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:17.284723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:54:17.326171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012432a3-e48b-473f-81e5-552e0d924817 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:17.330015 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ca617-c606-4008-992e-0c60b56d29e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:17.330580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ca617-c606-4008-992e-0c60b56d29e1 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:17.330873 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ca617-c606-4008-992e-0c60b56d29e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:17.331148 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ca617-c606-4008-992e-0c60b56d29e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:17.340392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ca617-c606-4008-992e-0c60b56d29e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:17.341824 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ca617-c606-4008-992e-0c60b56d29e1 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:17.351349 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1154/2319] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:54:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:54:21.530628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abf57e-bae3-47a3-8451-62a50e246e64 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:21.532828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-00abf57e-bae3-47a3-8451-62a50e246e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:21.555718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abf57e-bae3-47a3-8451-62a50e246e64 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:21.560616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ecd80-1040-4520-943f-b1d4ae308e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:21.561364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ecd80-1040-4520-943f-b1d4ae308e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:21.561716 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ecd80-1040-4520-943f-b1d4ae308e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:21.562055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ecd80-1040-4520-943f-b1d4ae308e8b admin admin] PATH_INFO: `/v3/users/ef11980490564c11aaa14404190c764d` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:21.563284 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ecd80-1040-4520-943f-b1d4ae308e8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef11980490564c11aaa14404190c764d)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:21.565334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ecd80-1040-4520-943f-b1d4ae308e8b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:21.609263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:54:21.641345 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bbb71-f3ec-447d-b5b9-7ba54226d1a7 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:21.642361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bbb71-f3ec-447d-b5b9-7ba54226d1a7 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:21.642961 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bbb71-f3ec-447d-b5b9-7ba54226d1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:21.657947 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-075ecd80-1040-4520-943f-b1d4ae308e8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef11980490564c11aaa14404190c764d'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:21.667934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-075ecd80-1040-4520-943f-b1d4ae308e8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef11980490564c11aaa14404190c764d'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:21.679155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-135bbb71-f3ec-447d-b5b9-7ba54226d1a7 None None] MFA Rules not processed for user `effc38ce5cd6441b92ed3bbb2eaee76c`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:54:21.680999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-135bbb71-f3ec-447d-b5b9-7ba54226d1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:21.692774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-075ecd80-1040-4520-943f-b1d4ae308e8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef11980490564c11aaa14404190c764d'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:21.704175 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-075ecd80-1040-4520-943f-b1d4ae308e8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef11980490564c11aaa14404190c764d'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:21.705058 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-075ecd80-1040-4520-943f-b1d4ae308e8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef11980490564c11aaa14404190c764d'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:21.711876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1166/2320] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:54:21 2022] DELETE /identity/v3/users/ef11980490564c11aaa14404190c764d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:54:21.717710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102639f-b3ae-4536-b428-8c2c3262999b None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:21.718141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102639f-b3ae-4536-b428-8c2c3262999b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:21.718567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102639f-b3ae-4536-b428-8c2c3262999b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:21.740261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-f102639f-b3ae-4536-b428-8c2c3262999b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:54:21.741594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f102639f-b3ae-4536-b428-8c2c3262999b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:21.768774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1155/2321] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:54:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:54:21.773397 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bbb71-f3ec-447d-b5b9-7ba54226d1a7 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:21.775687 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-135bbb71-f3ec-447d-b5b9-7ba54226d1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:21.791513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bbb71-f3ec-447d-b5b9-7ba54226d1a7 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:21.813073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1167/2322] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:54:21.823750 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102639f-b3ae-4536-b428-8c2c3262999b None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:21.825198 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f102639f-b3ae-4536-b428-8c2c3262999b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:21.866862 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f4a44-38df-4a01-b6b2-12438ea10c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:21.867542 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f4a44-38df-4a01-b6b2-12438ea10c71 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:21.868523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f4a44-38df-4a01-b6b2-12438ea10c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:21.868804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f4a44-38df-4a01-b6b2-12438ea10c71 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:21.869493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f4a44-38df-4a01-b6b2-12438ea10c71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:21.871453 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f4a44-38df-4a01-b6b2-12438ea10c71 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:21.901960 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102639f-b3ae-4536-b428-8c2c3262999b None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:21.905838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32409450-e8e3-4667-b031-115157c532c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:21.906492 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32409450-e8e3-4667-b031-115157c532c7 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:21.906775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32409450-e8e3-4667-b031-115157c532c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:21.907051 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32409450-e8e3-4667-b031-115157c532c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:21.907672 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1156/2323] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:54:21 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:54:21.916765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f4a44-38df-4a01-b6b2-12438ea10c71 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:21.926561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32409450-e8e3-4667-b031-115157c532c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:21.928032 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32409450-e8e3-4667-b031-115157c532c7 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:21.945374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1168/2324] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:54:21.970092 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:54:21.997534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f4a44-38df-4a01-b6b2-12438ea10c71 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:22.003287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729c9fb-9f12-4035-8cf4-de3cdebe217c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:22.004072 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729c9fb-9f12-4035-8cf4-de3cdebe217c admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:22.004523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729c9fb-9f12-4035-8cf4-de3cdebe217c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:22.004972 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729c9fb-9f12-4035-8cf4-de3cdebe217c admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:22.005899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729c9fb-9f12-4035-8cf4-de3cdebe217c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:22.008608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729c9fb-9f12-4035-8cf4-de3cdebe217c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:22.034005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1157/2325] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:54:21 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:54:22.038090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32409450-e8e3-4667-b031-115157c532c7 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:22.049001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32409450-e8e3-4667-b031-115157c532c7 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:22.054161 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3efe36-14a0-459a-ad54-1e1c5ade68c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:22.054886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3efe36-14a0-459a-ad54-1e1c5ade68c1 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:22.055307 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3efe36-14a0-459a-ad54-1e1c5ade68c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:22.055649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3efe36-14a0-459a-ad54-1e1c5ade68c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:22.056588 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3efe36-14a0-459a-ad54-1e1c5ade68c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:22.058342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3efe36-14a0-459a-ad54-1e1c5ade68c1 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:22.075091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1169/2326] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:54:22.077198 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729c9fb-9f12-4035-8cf4-de3cdebe217c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:22.091547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729c9fb-9f12-4035-8cf4-de3cdebe217c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:22.099416 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02a199-511d-407b-945d-b923f70ce796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:22.100265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02a199-511d-407b-945d-b923f70ce796 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:22.100678 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02a199-511d-407b-945d-b923f70ce796 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:22.101030 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02a199-511d-407b-945d-b923f70ce796 admin admin] PATH_INFO: `/v3/projects/0b9346f2f32d4733bec958c53797e88b/users/e446a9defd8e4b19aaeb590972b735e7/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:22.123548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e02a199-511d-407b-945d-b923f70ce796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9346f2f32d4733bec958c53797e88b, user_id=e446a9defd8e4b19aaeb590972b735e7, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:22.126662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e02a199-511d-407b-945d-b923f70ce796 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:22.170088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1158/2327] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:54:22 2022] PUT /identity/v3/projects/0b9346f2f32d4733bec958c53797e88b/users/e446a9defd8e4b19aaeb590972b735e7/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:54:22.173848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3efe36-14a0-459a-ad54-1e1c5ade68c1 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:22.195237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3efe36-14a0-459a-ad54-1e1c5ade68c1 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:22.237688 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:54:22.288026 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02a199-511d-407b-945d-b923f70ce796 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:22.292963 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-7e02a199-511d-407b-945d-b923f70ce796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:22.301770 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab270d-7f48-462b-a7fc-aa6b6d6aae79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:22.303423 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab270d-7f48-462b-a7fc-aa6b6d6aae79 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:22.303870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab270d-7f48-462b-a7fc-aa6b6d6aae79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:22.304354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab270d-7f48-462b-a7fc-aa6b6d6aae79 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:22.306636 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab270d-7f48-462b-a7fc-aa6b6d6aae79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:22.308715 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab270d-7f48-462b-a7fc-aa6b6d6aae79 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:22.322288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1170/2328] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:54:22.327262 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab270d-7f48-462b-a7fc-aa6b6d6aae79 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:22.332054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02a199-511d-407b-945d-b923f70ce796 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:22.343935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d274a-a88e-483d-96d5-550f75d0d6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:22.344588 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab270d-7f48-462b-a7fc-aa6b6d6aae79 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:22.344735 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d274a-a88e-483d-96d5-550f75d0d6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:22.344914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d274a-a88e-483d-96d5-550f75d0d6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:22.345194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d274a-a88e-483d-96d5-550f75d0d6c3 admin admin] PATH_INFO: `/v3/projects/b440acf4b71c46cc81602cd40c80ff8a` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:22.347306 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d274a-a88e-483d-96d5-550f75d0d6c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b440acf4b71c46cc81602cd40c80ff8a)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:22.349284 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d274a-a88e-483d-96d5-550f75d0d6c3 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:22.350437 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d584dc-8335-4c79-88a1-925304e462d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:22.351476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d584dc-8335-4c79-88a1-925304e462d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:22.352538 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d584dc-8335-4c79-88a1-925304e462d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:22.353029 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d584dc-8335-4c79-88a1-925304e462d9 admin admin] PATH_INFO: `/v3/projects/0b9346f2f32d4733bec958c53797e88b/users/e446a9defd8e4b19aaeb590972b735e7/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:22.355724 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d584dc-8335-4c79-88a1-925304e462d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9346f2f32d4733bec958c53797e88b, user_id=e446a9defd8e4b19aaeb590972b735e7, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:22.359211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d584dc-8335-4c79-88a1-925304e462d9 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:22.375743 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1171/2329] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:54:22 2022] PUT /identity/v3/projects/0b9346f2f32d4733bec958c53797e88b/users/e446a9defd8e4b19aaeb590972b735e7/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:54:22.414275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-2d8d274a-a88e-483d-96d5-550f75d0d6c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b440acf4b71c46cc81602cd40c80ff8a', 'request_id': 'req-2d8d274a-a88e-483d-96d5-550f75d0d6c3'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:22.420276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-2d8d274a-a88e-483d-96d5-550f75d0d6c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b440acf4b71c46cc81602cd40c80ff8a', 'request_id': 'req-2d8d274a-a88e-483d-96d5-550f75d0d6c3'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:22.427085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-2d8d274a-a88e-483d-96d5-550f75d0d6c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b440acf4b71c46cc81602cd40c80ff8a', 'request_id': 'req-2d8d274a-a88e-483d-96d5-550f75d0d6c3'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:22.448920 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-2d8d274a-a88e-483d-96d5-550f75d0d6c3 admin admin] The token cache is being invalidate because project b440acf4b71c46cc81602cd40c80ff8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:54:22.449312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-2d8d274a-a88e-483d-96d5-550f75d0d6c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:22.450082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-2d8d274a-a88e-483d-96d5-550f75d0d6c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:54:22.464341 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1159/2330] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:54:22 2022] DELETE /identity/v3/projects/b440acf4b71c46cc81602cd40c80ff8a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:54:27.124629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3f2a9-6c73-40a3-ae68-e1c382fd90ee None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:27.125625 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3f2a9-6c73-40a3-ae68-e1c382fd90ee None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:27.126188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3f2a9-6c73-40a3-ae68-e1c382fd90ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:27.205154 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-f5d3f2a9-6c73-40a3-ae68-e1c382fd90ee None None] MFA Rules not processed for user `e446a9defd8e4b19aaeb590972b735e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:54:27.205154 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f5d3f2a9-6c73-40a3-ae68-e1c382fd90ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:27.311683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1172/2331] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:54:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:54:27.324953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d274a-a88e-483d-96d5-550f75d0d6c3 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:27.326440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-2d8d274a-a88e-483d-96d5-550f75d0d6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:27.395753 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d274a-a88e-483d-96d5-550f75d0d6c3 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:27.399700 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace0cf1-97c0-4d8f-945d-e30baaf05820 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4799b0c9e5aa47d389121d2183666d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:27.400382 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace0cf1-97c0-4d8f-945d-e30baaf05820 service glance] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:27.400670 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace0cf1-97c0-4d8f-945d-e30baaf05820 service glance] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:27.400948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace0cf1-97c0-4d8f-945d-e30baaf05820 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:27.402816 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-dace0cf1-97c0-4d8f-945d-e30baaf05820 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:27.448786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:54:27.469948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace0cf1-97c0-4d8f-945d-e30baaf05820 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:27.471497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace0cf1-97c0-4d8f-945d-e30baaf05820 service glance] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:27.481665 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1160/2332] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:54:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:54:27.616132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56753c85-ae6d-4ff8-955e-dcbab3ab1cf8 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:27.616685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56753c85-ae6d-4ff8-955e-dcbab3ab1cf8 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:27.618603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56753c85-ae6d-4ff8-955e-dcbab3ab1cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:27.640897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-56753c85-ae6d-4ff8-955e-dcbab3ab1cf8 None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:54:27.642156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-56753c85-ae6d-4ff8-955e-dcbab3ab1cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:27.702661 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1173/2333] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Aug 3 18:54:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:54:27.725452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace0cf1-97c0-4d8f-945d-e30baaf05820 service glance] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:27.727595 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-dace0cf1-97c0-4d8f-945d-e30baaf05820 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:27.924600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace0cf1-97c0-4d8f-945d-e30baaf05820 service glance] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:27.929528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44b647-778f-4212-bd6b-cbed0aad1dd7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:27.930300 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44b647-778f-4212-bd6b-cbed0aad1dd7 service swift] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:27.930583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44b647-778f-4212-bd6b-cbed0aad1dd7 service swift] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:27.930861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44b647-778f-4212-bd6b-cbed0aad1dd7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:27.939928 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44b647-778f-4212-bd6b-cbed0aad1dd7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:27.941562 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44b647-778f-4212-bd6b-cbed0aad1dd7 service swift] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:27.949862 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1161/2334] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 18:54:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 18:54:37.031624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff20ec-2f22-4696-808e-5295325c31b8 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:37.032373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff20ec-2f22-4696-808e-5295325c31b8 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:37.032373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff20ec-2f22-4696-808e-5295325c31b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:37.053061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-b4ff20ec-2f22-4696-808e-5295325c31b8 None None] MFA Rules not processed for user `effc38ce5cd6441b92ed3bbb2eaee76c`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:54:37.054869 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b4ff20ec-2f22-4696-808e-5295325c31b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:37.062061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1174/2335] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:54:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:54:37.946731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=41 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:54:39.288980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e543bd-d659-487a-b00b-6ef14716f3b7 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:39.289929 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e543bd-d659-487a-b00b-6ef14716f3b7 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:39.290208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e543bd-d659-487a-b00b-6ef14716f3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:39.310424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-a3e543bd-d659-487a-b00b-6ef14716f3b7 None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:54:39.311629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a3e543bd-d659-487a-b00b-6ef14716f3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:39.319105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1162/2336] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Aug 3 18:54:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:54:39.331541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff20ec-2f22-4696-808e-5295325c31b8 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:54:39.343391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff20ec-2f22-4696-808e-5295325c31b8 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:54:39.348981 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803ebf2-e8d3-4efc-aed0-f4edfde48981 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:54:39.349697 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803ebf2-e8d3-4efc-aed0-f4edfde48981 service swift] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:39.350058 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803ebf2-e8d3-4efc-aed0-f4edfde48981 service swift] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:39.350437 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803ebf2-e8d3-4efc-aed0-f4edfde48981 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:39.360055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803ebf2-e8d3-4efc-aed0-f4edfde48981 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:54:39.361736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803ebf2-e8d3-4efc-aed0-f4edfde48981 service swift] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:54:39.371033 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1175/2337] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 18:54:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 18:54:44.763443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cedb3-5ca9-4e6a-8001-a6eee962b464 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:54:44.764012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cedb3-5ca9-4e6a-8001-a6eee962b464 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:54:44.764231 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cedb3-5ca9-4e6a-8001-a6eee962b464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:54:44.785761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-6d1cedb3-5ca9-4e6a-8001-a6eee962b464 None None] MFA Rules not processed for user `fe100dded4484be3981b33384f4c68b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:54:44.786969 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-6d1cedb3-5ca9-4e6a-8001-a6eee962b464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:54:44.842172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1163/2338] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:54:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:54:49.367353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=87,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:54:54.834470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=28 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:55:14.028237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803ebf2-e8d3-4efc-aed0-f4edfde48981 service swift] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:55:14.029724 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2803ebf2-e8d3-4efc-aed0-f4edfde48981 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:14.085826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803ebf2-e8d3-4efc-aed0-f4edfde48981 service swift] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:55:14.089468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055e33d-eb67-44dd-981d-16674db5f734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:55:14.090089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055e33d-eb67-44dd-981d-16674db5f734 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:14.090371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055e33d-eb67-44dd-981d-16674db5f734 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:14.090643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055e33d-eb67-44dd-981d-16674db5f734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:14.092461 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2055e33d-eb67-44dd-981d-16674db5f734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:14.165195 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055e33d-eb67-44dd-981d-16674db5f734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:55:14.166776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055e33d-eb67-44dd-981d-16674db5f734 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:55:14.178732 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1176/2339] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:55:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:55:14.269824 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cedb3-5ca9-4e6a-8001-a6eee962b464 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:55:14.280093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cedb3-5ca9-4e6a-8001-a6eee962b464 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:55:14.283782 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298a6cc-0343-48f4-b4da-bb7d11230310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:55:14.284432 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298a6cc-0343-48f4-b4da-bb7d11230310 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:14.284711 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298a6cc-0343-48f4-b4da-bb7d11230310 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:14.284980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298a6cc-0343-48f4-b4da-bb7d11230310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:14.287455 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b298a6cc-0343-48f4-b4da-bb7d11230310 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:14.306249 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298a6cc-0343-48f4-b4da-bb7d11230310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:55:14.307791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298a6cc-0343-48f4-b4da-bb7d11230310 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:55:14.319010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1164/2340] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:55:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:55:24.175525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=54 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:55:24.313856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=6 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:55:25.613342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b0d28-e6fc-4b66-8790-bdda169ce0cb None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:25.613898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b0d28-e6fc-4b66-8790-bdda169ce0cb None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:25.614293 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b0d28-e6fc-4b66-8790-bdda169ce0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:25.642201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-ce6b0d28-e6fc-4b66-8790-bdda169ce0cb None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:55:25.643411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ce6b0d28-e6fc-4b66-8790-bdda169ce0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:25.651528 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1177/2341] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Aug 3 18:55:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:55:25.663449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298a6cc-0343-48f4-b4da-bb7d11230310 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:55:25.676494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298a6cc-0343-48f4-b4da-bb7d11230310 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:55:25.682480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d634491-938b-4d54-8a97-15d85c83b473 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:55:25.683113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d634491-938b-4d54-8a97-15d85c83b473 service swift] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:25.683399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d634491-938b-4d54-8a97-15d85c83b473 service swift] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:25.683677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d634491-938b-4d54-8a97-15d85c83b473 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:25.692332 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d634491-938b-4d54-8a97-15d85c83b473 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:55:25.694132 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d634491-938b-4d54-8a97-15d85c83b473 service swift] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:55:25.703455 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1165/2342] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 18:55:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 18:55:30.238970 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b0d28-e6fc-4b66-8790-bdda169ce0cb None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:55:30.240397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ce6b0d28-e6fc-4b66-8790-bdda169ce0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:30.259005 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b0d28-e6fc-4b66-8790-bdda169ce0cb None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:55:30.265695 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221b0e9-df68-4fab-9f48-a867983761c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:55:30.266403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221b0e9-df68-4fab-9f48-a867983761c2 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:30.266747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221b0e9-df68-4fab-9f48-a867983761c2 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:30.267084 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221b0e9-df68-4fab-9f48-a867983761c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:30.269338 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-6221b0e9-df68-4fab-9f48-a867983761c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:30.357413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221b0e9-df68-4fab-9f48-a867983761c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:55:30.359035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221b0e9-df68-4fab-9f48-a867983761c2 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:55:30.369091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1178/2343] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:55:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:55:33.128432 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d634491-938b-4d54-8a97-15d85c83b473 service swift] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:55:33.139244 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d634491-938b-4d54-8a97-15d85c83b473 service swift] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:55:33.143020 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64d360-169f-4f20-90bc-fd02c8b52033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:55:33.143914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64d360-169f-4f20-90bc-fd02c8b52033 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:33.144447 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64d360-169f-4f20-90bc-fd02c8b52033 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:33.144881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64d360-169f-4f20-90bc-fd02c8b52033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:33.147366 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-af64d360-169f-4f20-90bc-fd02c8b52033 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:33.223742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64d360-169f-4f20-90bc-fd02c8b52033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:55:33.225372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64d360-169f-4f20-90bc-fd02c8b52033 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:55:33.236824 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1166/2344] 10.222.0.69 () {60 vars in 1396 bytes} [Wed Aug 3 18:55:33 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:55:40.364082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=31 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:55:40.788746 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221b0e9-df68-4fab-9f48-a867983761c2 service nova] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:55:40.801169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221b0e9-df68-4fab-9f48-a867983761c2 service nova] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:55:40.804924 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd779b-57fa-49e3-a9e4-aced3adac834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:55:40.805748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd779b-57fa-49e3-a9e4-aced3adac834 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:40.806021 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd779b-57fa-49e3-a9e4-aced3adac834 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:40.806300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd779b-57fa-49e3-a9e4-aced3adac834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:40.809141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-fadd779b-57fa-49e3-a9e4-aced3adac834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:40.820127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd779b-57fa-49e3-a9e4-aced3adac834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:55:40.821699 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd779b-57fa-49e3-a9e4-aced3adac834 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:55:40.831126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1179/2345] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:55:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:55:42.263796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64d360-169f-4f20-90bc-fd02c8b52033 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:55:42.265287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-af64d360-169f-4f20-90bc-fd02c8b52033 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:42.276030 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64d360-169f-4f20-90bc-fd02c8b52033 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:55:42.279691 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13164e98-7121-4745-95aa-29131e35a91e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:55:42.280329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13164e98-7121-4745-95aa-29131e35a91e service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:42.280617 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13164e98-7121-4745-95aa-29131e35a91e service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:42.280891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13164e98-7121-4745-95aa-29131e35a91e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:42.291731 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13164e98-7121-4745-95aa-29131e35a91e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:55:42.292404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13164e98-7121-4745-95aa-29131e35a91e service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:55:42.302409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1167/2346] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:55:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:55:45.779505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd779b-57fa-49e3-a9e4-aced3adac834 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:55:45.788258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd779b-57fa-49e3-a9e4-aced3adac834 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:55:45.794293 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c50e2c-f365-4ddc-8dc2-18cdd5a7906d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:55:45.795048 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c50e2c-f365-4ddc-8dc2-18cdd5a7906d service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:45.795339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c50e2c-f365-4ddc-8dc2-18cdd5a7906d service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:45.795614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c50e2c-f365-4ddc-8dc2-18cdd5a7906d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:45.797519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-88c50e2c-f365-4ddc-8dc2-18cdd5a7906d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:45.807963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c50e2c-f365-4ddc-8dc2-18cdd5a7906d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:55:45.809476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c50e2c-f365-4ddc-8dc2-18cdd5a7906d service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:55:45.818650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1180/2347] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:55:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:55:45.932454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13164e98-7121-4745-95aa-29131e35a91e service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:55:45.935531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-13164e98-7121-4745-95aa-29131e35a91e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:45.960514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13164e98-7121-4745-95aa-29131e35a91e service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:55:45.972222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e707e83-1351-41f9-9383-999ac0258d53 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:45.973302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e707e83-1351-41f9-9383-999ac0258d53 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:45.973302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e707e83-1351-41f9-9383-999ac0258d53 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:45.974140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1181/2348] 10.222.0.69 () {54 vars in 921 bytes} [Wed Aug 3 18:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 18:55:45.981729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df3119-3572-4247-959f-ea704626ffe1 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:45.982010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df3119-3572-4247-959f-ea704626ffe1 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:45.982567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df3119-3572-4247-959f-ea704626ffe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:45.988238 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c535d-c1c3-496e-879b-8b56fd605d66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:55:45.989600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c535d-c1c3-496e-879b-8b56fd605d66 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:45.990046 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c535d-c1c3-496e-879b-8b56fd605d66 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:45.990468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c535d-c1c3-496e-879b-8b56fd605d66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:45.993545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-de6c535d-c1c3-496e-879b-8b56fd605d66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:46.011350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c535d-c1c3-496e-879b-8b56fd605d66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:55:46.013420 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c535d-c1c3-496e-879b-8b56fd605d66 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:55:46.020137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-32df3119-3572-4247-959f-ea704626ffe1 None None] MFA Rules not processed for user `850a50cfe4394febbafb0908b08888b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:55:46.025739 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-32df3119-3572-4247-959f-ea704626ffe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:46.041677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1168/2349] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:55:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:55:46.044146 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1182/2350] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Aug 3 18:55:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:55:46.054353 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c535d-c1c3-496e-879b-8b56fd605d66 service nova] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:55:46.064575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c535d-c1c3-496e-879b-8b56fd605d66 service nova] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:55:46.068537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072afaeb-d3ac-4d6c-91a4-ebcb317d8aa8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4799b0c9e5aa47d389121d2183666d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:55:46.069337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072afaeb-d3ac-4d6c-91a4-ebcb317d8aa8 service glance] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:46.069767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072afaeb-d3ac-4d6c-91a4-ebcb317d8aa8 service glance] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:46.070363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072afaeb-d3ac-4d6c-91a4-ebcb317d8aa8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:46.089982 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072afaeb-d3ac-4d6c-91a4-ebcb317d8aa8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:55:46.091422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072afaeb-d3ac-4d6c-91a4-ebcb317d8aa8 service glance] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:55:46.104445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1169/2351] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:55:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:55:46.216013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7e2fd-d700-4eaf-9fd6-7fa3356b527c None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:46.216441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7e2fd-d700-4eaf-9fd6-7fa3356b527c None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:46.216794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7e2fd-d700-4eaf-9fd6-7fa3356b527c None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:46.218464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1183/2352] 10.222.0.69 () {54 vars in 921 bytes} [Wed Aug 3 18:55:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:55:46.224493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797ca8a-688e-44ca-995c-134df539a2e3 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:46.225168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797ca8a-688e-44ca-995c-134df539a2e3 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:46.225477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797ca8a-688e-44ca-995c-134df539a2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:46.252834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-2797ca8a-688e-44ca-995c-134df539a2e3 None None] MFA Rules not processed for user `0eb9ad672e5444098d71b7c9a27b72e8`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:55:46.254282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-2797ca8a-688e-44ca-995c-134df539a2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:46.265002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1170/2353] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Aug 3 18:55:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:55:46.280995 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7e2fd-d700-4eaf-9fd6-7fa3356b527c None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:55:46.295226 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7e2fd-d700-4eaf-9fd6-7fa3356b527c None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:55:46.299655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95632fae-6000-472a-9c12-73e6d3d89d76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:55:46.300569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95632fae-6000-472a-9c12-73e6d3d89d76 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:46.300921 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95632fae-6000-472a-9c12-73e6d3d89d76 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:46.301263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95632fae-6000-472a-9c12-73e6d3d89d76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:46.311245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95632fae-6000-472a-9c12-73e6d3d89d76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:55:46.312853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95632fae-6000-472a-9c12-73e6d3d89d76 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:55:46.324463 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1184/2354] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:55:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:55:46.419811 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71db30c-18ca-4041-b4f6-b38bbd8bce34 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:46.420124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71db30c-18ca-4041-b4f6-b38bbd8bce34 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:46.420441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71db30c-18ca-4041-b4f6-b38bbd8bce34 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:46.422127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1171/2355] 10.222.0.69 () {54 vars in 921 bytes} [Wed Aug 3 18:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 18:55:46.427581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033b385-c553-45a1-b2c2-d1dfdd9ffb61 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:46.427868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033b385-c553-45a1-b2c2-d1dfdd9ffb61 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:46.428144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033b385-c553-45a1-b2c2-d1dfdd9ffb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:46.449778 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-b033b385-c553-45a1-b2c2-d1dfdd9ffb61 None None] MFA Rules not processed for user `850a50cfe4394febbafb0908b08888b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:55:46.450983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b033b385-c553-45a1-b2c2-d1dfdd9ffb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:46.460034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1185/2356] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Aug 3 18:55:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:55:46.471373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71db30c-18ca-4041-b4f6-b38bbd8bce34 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:55:46.482703 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71db30c-18ca-4041-b4f6-b38bbd8bce34 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:55:46.486435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c8799-4d7b-4514-94de-02c17db2f31b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:55:46.487051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c8799-4d7b-4514-94de-02c17db2f31b service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:46.487503 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c8799-4d7b-4514-94de-02c17db2f31b service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:46.487775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c8799-4d7b-4514-94de-02c17db2f31b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:46.499296 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c8799-4d7b-4514-94de-02c17db2f31b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:55:46.501129 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c8799-4d7b-4514-94de-02c17db2f31b service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:55:46.514329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1172/2357] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:55:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:55:46.580141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033b385-c553-45a1-b2c2-d1dfdd9ffb61 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:55:46.581547 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b033b385-c553-45a1-b2c2-d1dfdd9ffb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:46.652521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033b385-c553-45a1-b2c2-d1dfdd9ffb61 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:55:46.657517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0bdec9-f3bc-4203-9e25-493a2f439118 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:55:46.658137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0bdec9-f3bc-4203-9e25-493a2f439118 service placement] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:46.658484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0bdec9-f3bc-4203-9e25-493a2f439118 service placement] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:46.658667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0bdec9-f3bc-4203-9e25-493a2f439118 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:46.660661 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7d0bdec9-f3bc-4203-9e25-493a2f439118 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:46.673134 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0bdec9-f3bc-4203-9e25-493a2f439118 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:55:46.674722 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0bdec9-f3bc-4203-9e25-493a2f439118 service placement] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:55:46.687730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1186/2358] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:55:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:55:50.362512 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b28ddd-335d-47b8-9a70-847c9c306a4a None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:50.363098 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b28ddd-335d-47b8-9a70-847c9c306a4a None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:50.363098 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b28ddd-335d-47b8-9a70-847c9c306a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:50.382690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-b7b28ddd-335d-47b8-9a70-847c9c306a4a None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:55:50.383968 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b7b28ddd-335d-47b8-9a70-847c9c306a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:55:50.391298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1173/2359] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Aug 3 18:55:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:55:50.403884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0bdec9-f3bc-4203-9e25-493a2f439118 service placement] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:55:50.414936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0bdec9-f3bc-4203-9e25-493a2f439118 service placement] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:55:50.420667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911aac83-13f9-407a-9e93-fcae2f4bf0ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:55:50.421279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aac83-13f9-407a-9e93-fcae2f4bf0ab service swift] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:55:50.421560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aac83-13f9-407a-9e93-fcae2f4bf0ab service swift] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:55:50.421835 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aac83-13f9-407a-9e93-fcae2f4bf0ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:55:50.429931 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911aac83-13f9-407a-9e93-fcae2f4bf0ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:55:50.431590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911aac83-13f9-407a-9e93-fcae2f4bf0ab service swift] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:55:50.439777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1187/2360] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 18:55:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 18:56:00.377819 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=64 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:56:00.436448 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=62 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:56:22.154220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b28ddd-335d-47b8-9a70-847c9c306a4a None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:56:22.156689 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b7b28ddd-335d-47b8-9a70-847c9c306a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:56:22.253555 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b28ddd-335d-47b8-9a70-847c9c306a4a None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:56:22.258067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1664bd-0068-4395-9b62-80939d2b521c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:56:22.258763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1664bd-0068-4395-9b62-80939d2b521c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:56:22.259045 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1664bd-0068-4395-9b62-80939d2b521c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:56:22.259327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1664bd-0068-4395-9b62-80939d2b521c admin admin] PATH_INFO: `/v3/users/fe100dded4484be3981b33384f4c68b2` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:56:22.260577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1664bd-0068-4395-9b62-80939d2b521c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe100dded4484be3981b33384f4c68b2)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:56:22.262519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1664bd-0068-4395-9b62-80939d2b521c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:56:22.319519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ef1664bd-0068-4395-9b62-80939d2b521c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe100dded4484be3981b33384f4c68b2'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:56:22.327621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ef1664bd-0068-4395-9b62-80939d2b521c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe100dded4484be3981b33384f4c68b2'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:56:22.341647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ef1664bd-0068-4395-9b62-80939d2b521c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe100dded4484be3981b33384f4c68b2'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:56:22.348776 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ef1664bd-0068-4395-9b62-80939d2b521c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe100dded4484be3981b33384f4c68b2'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:56:22.349561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ef1664bd-0068-4395-9b62-80939d2b521c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe100dded4484be3981b33384f4c68b2'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:56:22.356487 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1174/2361] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:56:22 2022] DELETE /identity/v3/users/fe100dded4484be3981b33384f4c68b2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:56:22.533678 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911aac83-13f9-407a-9e93-fcae2f4bf0ab service swift] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:56:22.545362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911aac83-13f9-407a-9e93-fcae2f4bf0ab service swift] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:56:22.606359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e471364-cfc0-4d5a-b3d1-bd6691de13f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:56:22.606914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e471364-cfc0-4d5a-b3d1-bd6691de13f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:56:22.607192 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e471364-cfc0-4d5a-b3d1-bd6691de13f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:56:22.607460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e471364-cfc0-4d5a-b3d1-bd6691de13f4 admin admin] PATH_INFO: `/v3/projects/653198c47b6a4f68b5ac4932b02b95d0` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:56:22.608628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e471364-cfc0-4d5a-b3d1-bd6691de13f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=653198c47b6a4f68b5ac4932b02b95d0)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:56:22.610519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e471364-cfc0-4d5a-b3d1-bd6691de13f4 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:56:22.661198 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0e471364-cfc0-4d5a-b3d1-bd6691de13f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '653198c47b6a4f68b5ac4932b02b95d0', 'request_id': 'req-0e471364-cfc0-4d5a-b3d1-bd6691de13f4'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:56:22.667839 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0e471364-cfc0-4d5a-b3d1-bd6691de13f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '653198c47b6a4f68b5ac4932b02b95d0', 'request_id': 'req-0e471364-cfc0-4d5a-b3d1-bd6691de13f4'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:56:22.674526 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0e471364-cfc0-4d5a-b3d1-bd6691de13f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '653198c47b6a4f68b5ac4932b02b95d0', 'request_id': 'req-0e471364-cfc0-4d5a-b3d1-bd6691de13f4'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:56:22.686314 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0e471364-cfc0-4d5a-b3d1-bd6691de13f4 admin admin] The token cache is being invalidate because project 653198c47b6a4f68b5ac4932b02b95d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:56:22.686720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0e471364-cfc0-4d5a-b3d1-bd6691de13f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:56:22.687362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0e471364-cfc0-4d5a-b3d1-bd6691de13f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:56:22.689326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1188/2362] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:56:22 2022] DELETE /identity/v3/projects/653198c47b6a4f68b5ac4932b02b95d0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:56:23.734284 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1664bd-0068-4395-9b62-80939d2b521c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:56:23.735806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ef1664bd-0068-4395-9b62-80939d2b521c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:56:23.833189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1664bd-0068-4395-9b62-80939d2b521c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:56:23.850924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:56:23.907094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc997cbd-f33a-4350-b849-13bac653a995 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:56:23.907708 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc997cbd-f33a-4350-b849-13bac653a995 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:56:23.907985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc997cbd-f33a-4350-b849-13bac653a995 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:56:23.908255 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc997cbd-f33a-4350-b849-13bac653a995 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:56:23.910044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-dc997cbd-f33a-4350-b849-13bac653a995 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:56:23.922012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc997cbd-f33a-4350-b849-13bac653a995 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:56:23.923579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc997cbd-f33a-4350-b849-13bac653a995 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:56:23.934472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1175/2363] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:56:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:56:27.594499 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd2ea4-d948-4d24-9122-ad20795ed277 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:56:27.594976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd2ea4-d948-4d24-9122-ad20795ed277 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:56:27.595092 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd2ea4-d948-4d24-9122-ad20795ed277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:56:27.616993 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-f4fd2ea4-d948-4d24-9122-ad20795ed277 None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:56:27.618275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f4fd2ea4-d948-4d24-9122-ad20795ed277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:56:27.676187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1189/2364] 10.222.0.69 () {58 vars in 967 bytes} [Wed Aug 3 18:56:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:56:27.699046 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc997cbd-f33a-4350-b849-13bac653a995 service nova] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:56:27.701231 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-dc997cbd-f33a-4350-b849-13bac653a995 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:56:27.772482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc997cbd-f33a-4350-b849-13bac653a995 service nova] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:56:27.777747 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f5a1c-3b43-4135-84b8-ebc354f4f107 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:56:27.778378 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f5a1c-3b43-4135-84b8-ebc354f4f107 service swift] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:56:27.778662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f5a1c-3b43-4135-84b8-ebc354f4f107 service swift] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:56:27.778938 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f5a1c-3b43-4135-84b8-ebc354f4f107 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:56:27.791516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f5a1c-3b43-4135-84b8-ebc354f4f107 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:56:27.793195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f5a1c-3b43-4135-84b8-ebc354f4f107 service swift] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:56:27.801817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1176/2365] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 18:56:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 18:56:29.989770 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd2ea4-d948-4d24-9122-ad20795ed277 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:56:29.991209 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f4fd2ea4-d948-4d24-9122-ad20795ed277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:56:30.007464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd2ea4-d948-4d24-9122-ad20795ed277 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:56:30.012977 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d797478-9205-4e72-b427-37ca60ba9786 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:56:30.013615 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d797478-9205-4e72-b427-37ca60ba9786 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:56:30.013891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d797478-9205-4e72-b427-37ca60ba9786 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:56:30.014161 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d797478-9205-4e72-b427-37ca60ba9786 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:56:30.015876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0d797478-9205-4e72-b427-37ca60ba9786 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:56:30.049395 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:56:30.076696 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d797478-9205-4e72-b427-37ca60ba9786 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:56:30.078298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d797478-9205-4e72-b427-37ca60ba9786 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:56:30.087451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1190/2366] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:56:29 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:56:37.799001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=58 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:56:40.083168 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=12 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:56:53.603948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380105d-1fd5-49ad-9ae0-2276e506a942 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:56:53.604577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380105d-1fd5-49ad-9ae0-2276e506a942 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:56:53.604699 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380105d-1fd5-49ad-9ae0-2276e506a942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:56:53.624262 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-1380105d-1fd5-49ad-9ae0-2276e506a942 None None] MFA Rules not processed for user `c6085ba9dff1480a8a1674e492e0c6c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:56:53.625525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1380105d-1fd5-49ad-9ae0-2276e506a942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:56:53.632715 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1177/2367] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:56:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:57:03.618562 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:57:53.982614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea635d4-966c-45d9-a84d-92bcc608ec9f None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:57:53.983127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea635d4-966c-45d9-a84d-92bcc608ec9f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:57:53.983230 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea635d4-966c-45d9-a84d-92bcc608ec9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:57:54.003666 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-2ea635d4-966c-45d9-a84d-92bcc608ec9f None None] MFA Rules not processed for user `1aa1d56d9b154e32aa1c136b577d49f4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:57:54.004899 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2ea635d4-966c-45d9-a84d-92bcc608ec9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:57:54.058234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1191/2368] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:57:53 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:57:54.370302 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380105d-1fd5-49ad-9ae0-2276e506a942 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:57:54.371801 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1380105d-1fd5-49ad-9ae0-2276e506a942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:57:54.428593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380105d-1fd5-49ad-9ae0-2276e506a942 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:57:54.432039 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4517da-3b6d-4fb2-8df2-cd574d99acae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:57:54.432691 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4517da-3b6d-4fb2-8df2-cd574d99acae service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:57:54.432976 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4517da-3b6d-4fb2-8df2-cd574d99acae service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:57:54.433245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4517da-3b6d-4fb2-8df2-cd574d99acae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:57:54.434958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ca4517da-3b6d-4fb2-8df2-cd574d99acae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:57:54.444575 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4517da-3b6d-4fb2-8df2-cd574d99acae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:57:54.445993 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4517da-3b6d-4fb2-8df2-cd574d99acae service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:57:54.454148 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1178/2369] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:57:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:58:04.051796 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=25 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:04.450497 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=26 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:08.232386 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea635d4-966c-45d9-a84d-92bcc608ec9f None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:08.233803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2ea635d4-966c-45d9-a84d-92bcc608ec9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:08.249685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea635d4-966c-45d9-a84d-92bcc608ec9f None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:08.255630 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d8b6b-9db0-4247-8d02-e273fde14dcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:08.256252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d8b6b-9db0-4247-8d02-e273fde14dcb service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:08.256563 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d8b6b-9db0-4247-8d02-e273fde14dcb service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:08.256837 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d8b6b-9db0-4247-8d02-e273fde14dcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:08.258576 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-dc4d8b6b-9db0-4247-8d02-e273fde14dcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:08.271467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d8b6b-9db0-4247-8d02-e273fde14dcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:08.273184 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d8b6b-9db0-4247-8d02-e273fde14dcb service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:08.284507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1192/2370] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:58:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:58:18.279242 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=6 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:22.776650 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4517da-3b6d-4fb2-8df2-cd574d99acae service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:22.778090 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ca4517da-3b6d-4fb2-8df2-cd574d99acae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:22.824179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4517da-3b6d-4fb2-8df2-cd574d99acae service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:22.827892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3211a9-afa4-478f-9c99-fb01e4652895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:22.828610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3211a9-afa4-478f-9c99-fb01e4652895 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:22.828900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3211a9-afa4-478f-9c99-fb01e4652895 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:22.829203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3211a9-afa4-478f-9c99-fb01e4652895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:22.831360 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-7a3211a9-afa4-478f-9c99-fb01e4652895 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:22.953417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3211a9-afa4-478f-9c99-fb01e4652895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:22.955470 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3211a9-afa4-478f-9c99-fb01e4652895 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:22.974725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1179/2371] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:58:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:58:26.586964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d8b6b-9db0-4247-8d02-e273fde14dcb service nova] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:26.588691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-dc4d8b6b-9db0-4247-8d02-e273fde14dcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:26.646625 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d8b6b-9db0-4247-8d02-e273fde14dcb service nova] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:26.651590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896bee35-0d7a-4202-9578-8f1aeaccd694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:26.652955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bee35-0d7a-4202-9578-8f1aeaccd694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:26.653315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bee35-0d7a-4202-9578-8f1aeaccd694 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:26.653672 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bee35-0d7a-4202-9578-8f1aeaccd694 admin admin] PATH_INFO: `/v3/users/c6085ba9dff1480a8a1674e492e0c6c3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:26.654883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896bee35-0d7a-4202-9578-8f1aeaccd694 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6085ba9dff1480a8a1674e492e0c6c3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:26.656917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896bee35-0d7a-4202-9578-8f1aeaccd694 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:26.704731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-896bee35-0d7a-4202-9578-8f1aeaccd694 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6085ba9dff1480a8a1674e492e0c6c3'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:26.712826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-896bee35-0d7a-4202-9578-8f1aeaccd694 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6085ba9dff1480a8a1674e492e0c6c3'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:26.725013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-896bee35-0d7a-4202-9578-8f1aeaccd694 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6085ba9dff1480a8a1674e492e0c6c3'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:26.731798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-896bee35-0d7a-4202-9578-8f1aeaccd694 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6085ba9dff1480a8a1674e492e0c6c3'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:26.732484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-896bee35-0d7a-4202-9578-8f1aeaccd694 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6085ba9dff1480a8a1674e492e0c6c3'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:26.739175 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1193/2372] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:58:26 2022] DELETE /identity/v3/users/c6085ba9dff1480a8a1674e492e0c6c3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:26.914454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3211a9-afa4-478f-9c99-fb01e4652895 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:26.925921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3211a9-afa4-478f-9c99-fb01e4652895 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:26.992334 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1098e8f-bc01-4faa-8b0f-8a843c9929b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:26.993018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1098e8f-bc01-4faa-8b0f-8a843c9929b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:26.993300 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1098e8f-bc01-4faa-8b0f-8a843c9929b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:26.993576 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1098e8f-bc01-4faa-8b0f-8a843c9929b5 admin admin] PATH_INFO: `/v3/projects/81ddb9c7b0f442df80831cc49287fd57` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:26.994734 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1098e8f-bc01-4faa-8b0f-8a843c9929b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ddb9c7b0f442df80831cc49287fd57)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:26.996675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1098e8f-bc01-4faa-8b0f-8a843c9929b5 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:27.017839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:27.043519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e1098e8f-bc01-4faa-8b0f-8a843c9929b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ddb9c7b0f442df80831cc49287fd57', 'request_id': 'req-e1098e8f-bc01-4faa-8b0f-8a843c9929b5'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:27.049869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e1098e8f-bc01-4faa-8b0f-8a843c9929b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ddb9c7b0f442df80831cc49287fd57', 'request_id': 'req-e1098e8f-bc01-4faa-8b0f-8a843c9929b5'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:27.055974 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e1098e8f-bc01-4faa-8b0f-8a843c9929b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ddb9c7b0f442df80831cc49287fd57', 'request_id': 'req-e1098e8f-bc01-4faa-8b0f-8a843c9929b5'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:27.068397 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e1098e8f-bc01-4faa-8b0f-8a843c9929b5 admin admin] The token cache is being invalidate because project 81ddb9c7b0f442df80831cc49287fd57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:58:27.068690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e1098e8f-bc01-4faa-8b0f-8a843c9929b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:27.069323 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e1098e8f-bc01-4faa-8b0f-8a843c9929b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:27.070877 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1180/2373] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:58:26 2022] DELETE /identity/v3/projects/81ddb9c7b0f442df80831cc49287fd57 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:27.076066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2da89-d130-4969-a63a-b0a37209f136 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:27.076386 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2da89-d130-4969-a63a-b0a37209f136 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:27.076669 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2da89-d130-4969-a63a-b0a37209f136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:27.134786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-29a2da89-d130-4969-a63a-b0a37209f136 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:27.135917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-29a2da89-d130-4969-a63a-b0a37209f136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:27.197316 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1194/2374] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:58:27.205760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f57197-18a5-4570-b08c-7f9d09e1e017 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:27.206050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f57197-18a5-4570-b08c-7f9d09e1e017 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:27.206331 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f57197-18a5-4570-b08c-7f9d09e1e017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:27.226073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-21f57197-18a5-4570-b08c-7f9d09e1e017 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:27.227271 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-21f57197-18a5-4570-b08c-7f9d09e1e017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:27.235947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1181/2375] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:58:27.240386 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2da89-d130-4969-a63a-b0a37209f136 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:27.250008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2da89-d130-4969-a63a-b0a37209f136 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:27.264573 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b08a82-7756-4777-8bba-99fdc25fd836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:27.265256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b08a82-7756-4777-8bba-99fdc25fd836 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:27.265566 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b08a82-7756-4777-8bba-99fdc25fd836 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:27.266851 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b08a82-7756-4777-8bba-99fdc25fd836 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:27.268259 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b08a82-7756-4777-8bba-99fdc25fd836 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:58:27.268259 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b08a82-7756-4777-8bba-99fdc25fd836 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:27.269423 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b08a82-7756-4777-8bba-99fdc25fd836 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:27.272090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:27.286355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1195/2376] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:58:27.291455 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f57197-18a5-4570-b08c-7f9d09e1e017 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:27.303706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f57197-18a5-4570-b08c-7f9d09e1e017 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:27.308372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003ff7c-1e29-49f4-8d93-2959711605b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:27.308989 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003ff7c-1e29-49f4-8d93-2959711605b4 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:27.309298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003ff7c-1e29-49f4-8d93-2959711605b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:27.309572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003ff7c-1e29-49f4-8d93-2959711605b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:27.310251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003ff7c-1e29-49f4-8d93-2959711605b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:27.312184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003ff7c-1e29-49f4-8d93-2959711605b4 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:27.342543 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1182/2377] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:58:27 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:58:27.348048 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b08a82-7756-4777-8bba-99fdc25fd836 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:27.358012 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b08a82-7756-4777-8bba-99fdc25fd836 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:27.417039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfe848-12bf-47d2-bec5-d4aef8f008fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:27.417577 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfe848-12bf-47d2-bec5-d4aef8f008fa admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:27.417857 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfe848-12bf-47d2-bec5-d4aef8f008fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:27.418130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfe848-12bf-47d2-bec5-d4aef8f008fa admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:27.418794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfe848-12bf-47d2-bec5-d4aef8f008fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:27.420740 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfe848-12bf-47d2-bec5-d4aef8f008fa admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:27.441596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1196/2378] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:58:27 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:58:27.445682 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003ff7c-1e29-49f4-8d93-2959711605b4 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:27.456671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003ff7c-1e29-49f4-8d93-2959711605b4 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:27.461528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f69de2-0ccb-4138-9693-78ad7b16370e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:27.462212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f69de2-0ccb-4138-9693-78ad7b16370e admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:27.462496 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f69de2-0ccb-4138-9693-78ad7b16370e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:27.462761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f69de2-0ccb-4138-9693-78ad7b16370e admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:27.463505 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f69de2-0ccb-4138-9693-78ad7b16370e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:27.465096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f69de2-0ccb-4138-9693-78ad7b16370e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:27.476748 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1183/2379] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:58:27.480914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfe848-12bf-47d2-bec5-d4aef8f008fa admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:27.490866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfe848-12bf-47d2-bec5-d4aef8f008fa admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:27.495463 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b3e5d-e22f-4440-a8a7-ab1cd24eb510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:27.496070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b3e5d-e22f-4440-a8a7-ab1cd24eb510 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:27.496392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b3e5d-e22f-4440-a8a7-ab1cd24eb510 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:27.496670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b3e5d-e22f-4440-a8a7-ab1cd24eb510 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:27.497363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b3e5d-e22f-4440-a8a7-ab1cd24eb510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:27.498959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b3e5d-e22f-4440-a8a7-ab1cd24eb510 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:27.509440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1197/2380] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:58:27.513550 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f69de2-0ccb-4138-9693-78ad7b16370e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:27.524488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f69de2-0ccb-4138-9693-78ad7b16370e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:27.529101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4829101-9c41-438f-b485-f34367ea66ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:27.529757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4829101-9c41-438f-b485-f34367ea66ed admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:27.530036 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4829101-9c41-438f-b485-f34367ea66ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:27.530311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4829101-9c41-438f-b485-f34367ea66ed admin admin] PATH_INFO: `/v3/projects/3fbd942e274a4c83a5ea3fe3f0b08c9b/users/e411fd6d51164ed79af6bf7e4b51bb2e/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:27.548618 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4829101-9c41-438f-b485-f34367ea66ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fbd942e274a4c83a5ea3fe3f0b08c9b, user_id=e411fd6d51164ed79af6bf7e4b51bb2e, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:27.551272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4829101-9c41-438f-b485-f34367ea66ed admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:27.564079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1184/2381] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:58:27 2022] PUT /identity/v3/projects/3fbd942e274a4c83a5ea3fe3f0b08c9b/users/e411fd6d51164ed79af6bf7e4b51bb2e/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:27.575560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a954768-de8a-4bf7-aefd-969569d10989 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:27.575912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a954768-de8a-4bf7-aefd-969569d10989 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:27.576265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a954768-de8a-4bf7-aefd-969569d10989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:27.614732 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-0a954768-de8a-4bf7-aefd-969569d10989 None None] MFA Rules not processed for user `e411fd6d51164ed79af6bf7e4b51bb2e`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:27.615978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0a954768-de8a-4bf7-aefd-969569d10989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:27.685391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1198/2382] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:58:27.697299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4829101-9c41-438f-b485-f34367ea66ed admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:27.699119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f4829101-9c41-438f-b485-f34367ea66ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:27.799641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4829101-9c41-438f-b485-f34367ea66ed admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:27.805240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154c01d-4762-4303-b5e2-00323e74d2ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:27.805883 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154c01d-4762-4303-b5e2-00323e74d2ec service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:27.806170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154c01d-4762-4303-b5e2-00323e74d2ec service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:27.806438 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154c01d-4762-4303-b5e2-00323e74d2ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:27.815132 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154c01d-4762-4303-b5e2-00323e74d2ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:27.816839 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154c01d-4762-4303-b5e2-00323e74d2ec service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:27.825691 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1185/2383] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:58:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:58:30.528582 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a954768-de8a-4bf7-aefd-969569d10989 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:30.592667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a954768-de8a-4bf7-aefd-969569d10989 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:30.598845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e117612-d7de-42f6-949c-2463e47f1970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:30.599511 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e117612-d7de-42f6-949c-2463e47f1970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:30.599792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e117612-d7de-42f6-949c-2463e47f1970 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:30.600074 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e117612-d7de-42f6-949c-2463e47f1970 admin admin] PATH_INFO: `/v3/users/e411fd6d51164ed79af6bf7e4b51bb2e` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:30.601309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e117612-d7de-42f6-949c-2463e47f1970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e411fd6d51164ed79af6bf7e4b51bb2e)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:30.603258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e117612-d7de-42f6-949c-2463e47f1970 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:30.608571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:30.666176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5e117612-d7de-42f6-949c-2463e47f1970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e411fd6d51164ed79af6bf7e4b51bb2e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:30.675723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5e117612-d7de-42f6-949c-2463e47f1970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e411fd6d51164ed79af6bf7e4b51bb2e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:30.689238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5e117612-d7de-42f6-949c-2463e47f1970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e411fd6d51164ed79af6bf7e4b51bb2e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:30.696248 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5e117612-d7de-42f6-949c-2463e47f1970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e411fd6d51164ed79af6bf7e4b51bb2e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:30.696984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5e117612-d7de-42f6-949c-2463e47f1970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e411fd6d51164ed79af6bf7e4b51bb2e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:30.704932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1199/2384] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:58:30 2022] DELETE /identity/v3/users/e411fd6d51164ed79af6bf7e4b51bb2e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:30.710256 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cf393-807c-45c0-b930-d5d6fd03a5fe None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:30.710737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cf393-807c-45c0-b930-d5d6fd03a5fe None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:30.710903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cf393-807c-45c0-b930-d5d6fd03a5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:30.719355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:30.738794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-954cf393-807c-45c0-b930-d5d6fd03a5fe None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:30.740337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-954cf393-807c-45c0-b930-d5d6fd03a5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:30.808970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1186/2385] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:58:30.819578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e117612-d7de-42f6-949c-2463e47f1970 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:30.821130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5e117612-d7de-42f6-949c-2463e47f1970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:30.900117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e117612-d7de-42f6-949c-2463e47f1970 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:30.903845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fdbea-83e3-4abb-aa68-a93e9febce6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:30.904502 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fdbea-83e3-4abb-aa68-a93e9febce6a service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:30.904973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fdbea-83e3-4abb-aa68-a93e9febce6a service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:30.905460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fdbea-83e3-4abb-aa68-a93e9febce6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:30.917107 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415fdbea-83e3-4abb-aa68-a93e9febce6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:30.919474 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415fdbea-83e3-4abb-aa68-a93e9febce6a service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:30.935431 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1200/2386] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:58:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:58:31.215101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cf393-807c-45c0-b930-d5d6fd03a5fe None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:31.216601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-954cf393-807c-45c0-b930-d5d6fd03a5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:31.229079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cf393-807c-45c0-b930-d5d6fd03a5fe None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:31.233789 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f332a7-2baa-4d77-8640-acf638dac83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:31.234502 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f332a7-2baa-4d77-8640-acf638dac83e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:31.234773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f332a7-2baa-4d77-8640-acf638dac83e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:31.235376 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f332a7-2baa-4d77-8640-acf638dac83e admin admin] PATH_INFO: `/v3/projects/3fbd942e274a4c83a5ea3fe3f0b08c9b` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:31.236573 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f332a7-2baa-4d77-8640-acf638dac83e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fbd942e274a4c83a5ea3fe3f0b08c9b)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:31.240588 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f332a7-2baa-4d77-8640-acf638dac83e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:31.296557 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-36f332a7-2baa-4d77-8640-acf638dac83e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fbd942e274a4c83a5ea3fe3f0b08c9b', 'request_id': 'req-36f332a7-2baa-4d77-8640-acf638dac83e'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:31.302125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-36f332a7-2baa-4d77-8640-acf638dac83e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fbd942e274a4c83a5ea3fe3f0b08c9b', 'request_id': 'req-36f332a7-2baa-4d77-8640-acf638dac83e'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:31.308334 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-36f332a7-2baa-4d77-8640-acf638dac83e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fbd942e274a4c83a5ea3fe3f0b08c9b', 'request_id': 'req-36f332a7-2baa-4d77-8640-acf638dac83e'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:31.322219 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-36f332a7-2baa-4d77-8640-acf638dac83e admin admin] The token cache is being invalidate because project 3fbd942e274a4c83a5ea3fe3f0b08c9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:58:31.322511 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-36f332a7-2baa-4d77-8640-acf638dac83e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:31.323141 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-36f332a7-2baa-4d77-8640-acf638dac83e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:31.324712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1187/2387] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:58:31 2022] DELETE /identity/v3/projects/3fbd942e274a4c83a5ea3fe3f0b08c9b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:31.330496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db67351-8418-411f-867b-a8cb41b565ef None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:31.330789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db67351-8418-411f-867b-a8cb41b565ef None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:31.331090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db67351-8418-411f-867b-a8cb41b565ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:31.351117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-0db67351-8418-411f-867b-a8cb41b565ef None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:31.352351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0db67351-8418-411f-867b-a8cb41b565ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:31.412943 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1201/2388] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:58:31.421312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5ac6a-03eb-4eab-aba8-c85716e0de3b None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:31.421600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5ac6a-03eb-4eab-aba8-c85716e0de3b None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:31.421887 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5ac6a-03eb-4eab-aba8-c85716e0de3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:31.442043 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-e9a5ac6a-03eb-4eab-aba8-c85716e0de3b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:31.443265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e9a5ac6a-03eb-4eab-aba8-c85716e0de3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:31.451312 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1188/2389] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:58:31.456226 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db67351-8418-411f-867b-a8cb41b565ef None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:31.465973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db67351-8418-411f-867b-a8cb41b565ef None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:31.471054 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84e674-94ed-405a-9590-001fc1797e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:31.471677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84e674-94ed-405a-9590-001fc1797e7a admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:31.471956 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84e674-94ed-405a-9590-001fc1797e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:31.472233 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84e674-94ed-405a-9590-001fc1797e7a admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:31.472972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84e674-94ed-405a-9590-001fc1797e7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:58:31.473282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84e674-94ed-405a-9590-001fc1797e7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:31.475011 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84e674-94ed-405a-9590-001fc1797e7a admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:31.487880 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1202/2390] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:58:31.492689 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5ac6a-03eb-4eab-aba8-c85716e0de3b None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:31.503173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5ac6a-03eb-4eab-aba8-c85716e0de3b None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:31.507820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227b1e3-c783-4884-aafe-e7f45bd0ff0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:31.508466 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227b1e3-c783-4884-aafe-e7f45bd0ff0b admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:31.508796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227b1e3-c783-4884-aafe-e7f45bd0ff0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:31.509073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227b1e3-c783-4884-aafe-e7f45bd0ff0b admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:31.509734 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227b1e3-c783-4884-aafe-e7f45bd0ff0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:31.511604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227b1e3-c783-4884-aafe-e7f45bd0ff0b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:31.541136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1189/2391] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:58:31 2022] POST /identity/v3/projects => generated 357 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:58:31.545951 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84e674-94ed-405a-9590-001fc1797e7a admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:31.556884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84e674-94ed-405a-9590-001fc1797e7a admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:31.566157 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:31.614478 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec9f66-7c87-4d3d-aece-0544df9dc29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:31.615107 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec9f66-7c87-4d3d-aece-0544df9dc29f admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:31.615388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec9f66-7c87-4d3d-aece-0544df9dc29f admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:31.615666 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec9f66-7c87-4d3d-aece-0544df9dc29f admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:31.616356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec9f66-7c87-4d3d-aece-0544df9dc29f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:31.618284 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec9f66-7c87-4d3d-aece-0544df9dc29f admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:31.643060 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1203/2392] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:58:31 2022] POST /identity/v3/users => generated 279 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:58:31.647974 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227b1e3-c783-4884-aafe-e7f45bd0ff0b admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:31.659466 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227b1e3-c783-4884-aafe-e7f45bd0ff0b admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:31.664338 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abcd8d-47db-4cb4-b4e8-9eab0c6edeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:31.664904 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abcd8d-47db-4cb4-b4e8-9eab0c6edeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:31.665189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abcd8d-47db-4cb4-b4e8-9eab0c6edeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:31.665464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abcd8d-47db-4cb4-b4e8-9eab0c6edeb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:31.666162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abcd8d-47db-4cb4-b4e8-9eab0c6edeb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:31.667775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abcd8d-47db-4cb4-b4e8-9eab0c6edeb3 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:31.678686 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1190/2393] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:58:31.682952 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec9f66-7c87-4d3d-aece-0544df9dc29f admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:31.693765 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec9f66-7c87-4d3d-aece-0544df9dc29f admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:31.699936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f594d0d-6156-4f82-9ed8-bebfbe1bc0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:31.700593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f594d0d-6156-4f82-9ed8-bebfbe1bc0aa admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:31.700874 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f594d0d-6156-4f82-9ed8-bebfbe1bc0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:31.701148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f594d0d-6156-4f82-9ed8-bebfbe1bc0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:31.701848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f594d0d-6156-4f82-9ed8-bebfbe1bc0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:31.703481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f594d0d-6156-4f82-9ed8-bebfbe1bc0aa admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:31.714512 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1204/2394] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:58:31.718671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abcd8d-47db-4cb4-b4e8-9eab0c6edeb3 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:31.728907 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abcd8d-47db-4cb4-b4e8-9eab0c6edeb3 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:31.735738 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85542b6e-acb0-4256-b82b-3d62dae14e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:31.736445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85542b6e-acb0-4256-b82b-3d62dae14e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:31.736733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85542b6e-acb0-4256-b82b-3d62dae14e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:31.737011 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85542b6e-acb0-4256-b82b-3d62dae14e6c admin admin] PATH_INFO: `/v3/projects/fcd0043f356b4c85bbc4f02f99180aa7/users/79d818c7df654ec2ac805a6579e905c6/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:31.754077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85542b6e-acb0-4256-b82b-3d62dae14e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd0043f356b4c85bbc4f02f99180aa7, user_id=79d818c7df654ec2ac805a6579e905c6, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:31.757111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85542b6e-acb0-4256-b82b-3d62dae14e6c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:31.772637 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1191/2395] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:58:31 2022] PUT /identity/v3/projects/fcd0043f356b4c85bbc4f02f99180aa7/users/79d818c7df654ec2ac805a6579e905c6/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:31.787439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4a502-9938-4140-ac0e-e62c8cbdd382 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:31.787728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4a502-9938-4140-ac0e-e62c8cbdd382 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:31.788006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4a502-9938-4140-ac0e-e62c8cbdd382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:31.819563 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-3da4a502-9938-4140-ac0e-e62c8cbdd382 None None] MFA Rules not processed for user `79d818c7df654ec2ac805a6579e905c6`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:31.820821 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-3da4a502-9938-4140-ac0e-e62c8cbdd382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:31.886154 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1205/2396] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:58:31.890698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85542b6e-acb0-4256-b82b-3d62dae14e6c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:31.917005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:31.961521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85542b6e-acb0-4256-b82b-3d62dae14e6c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:31.966254 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420d6ac-36d3-462c-bf86-9be1a62a40a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:31.966871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420d6ac-36d3-462c-bf86-9be1a62a40a0 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:31.967156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420d6ac-36d3-462c-bf86-9be1a62a40a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:31.967434 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420d6ac-36d3-462c-bf86-9be1a62a40a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:31.968113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420d6ac-36d3-462c-bf86-9be1a62a40a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:31.971068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420d6ac-36d3-462c-bf86-9be1a62a40a0 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:31.998440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1192/2397] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:58:31 2022] POST /identity/v3/projects => generated 357 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:58:32.003253 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4a502-9938-4140-ac0e-e62c8cbdd382 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:32.013961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4a502-9938-4140-ac0e-e62c8cbdd382 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:32.071155 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c104e43-6e2e-4627-82d6-7ec7dd5191a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:32.071768 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c104e43-6e2e-4627-82d6-7ec7dd5191a6 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:32.072049 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c104e43-6e2e-4627-82d6-7ec7dd5191a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:32.072352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c104e43-6e2e-4627-82d6-7ec7dd5191a6 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:32.073010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c104e43-6e2e-4627-82d6-7ec7dd5191a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:32.075090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c104e43-6e2e-4627-82d6-7ec7dd5191a6 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:32.096233 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1206/2398] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:58:32 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:58:32.100219 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420d6ac-36d3-462c-bf86-9be1a62a40a0 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:32.110992 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420d6ac-36d3-462c-bf86-9be1a62a40a0 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:32.115655 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96485441-72ad-43eb-8fa9-698432bdf726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:32.116265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96485441-72ad-43eb-8fa9-698432bdf726 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:32.116577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96485441-72ad-43eb-8fa9-698432bdf726 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:32.116851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96485441-72ad-43eb-8fa9-698432bdf726 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:32.117549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96485441-72ad-43eb-8fa9-698432bdf726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:32.119135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96485441-72ad-43eb-8fa9-698432bdf726 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:32.130181 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1193/2399] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:58:32.134261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c104e43-6e2e-4627-82d6-7ec7dd5191a6 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:32.144165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c104e43-6e2e-4627-82d6-7ec7dd5191a6 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:32.148855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa6bd1-39f7-4ead-9300-5fedcda2b56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:32.149511 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa6bd1-39f7-4ead-9300-5fedcda2b56c admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:32.149796 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa6bd1-39f7-4ead-9300-5fedcda2b56c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:32.150067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa6bd1-39f7-4ead-9300-5fedcda2b56c admin admin] PATH_INFO: `/v3/projects/c68cf8daa1f74af7972c41c17f4720e8/users/45a7d3831bff4619aa3ef2d5dcc3f1f7/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:32.159251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:32.166179 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa6bd1-39f7-4ead-9300-5fedcda2b56c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68cf8daa1f74af7972c41c17f4720e8, user_id=45a7d3831bff4619aa3ef2d5dcc3f1f7, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:32.169154 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa6bd1-39f7-4ead-9300-5fedcda2b56c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:32.183168 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1207/2400] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:58:32 2022] PUT /identity/v3/projects/c68cf8daa1f74af7972c41c17f4720e8/users/45a7d3831bff4619aa3ef2d5dcc3f1f7/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:32.187217 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96485441-72ad-43eb-8fa9-698432bdf726 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:32.197480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96485441-72ad-43eb-8fa9-698432bdf726 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:32.261671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcfd8c-682a-45dc-9cd1-ffad94f41e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:32.262291 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcfd8c-682a-45dc-9cd1-ffad94f41e71 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:32.262574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcfd8c-682a-45dc-9cd1-ffad94f41e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:32.262851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcfd8c-682a-45dc-9cd1-ffad94f41e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:32.263541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcfd8c-682a-45dc-9cd1-ffad94f41e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:32.265154 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcfd8c-682a-45dc-9cd1-ffad94f41e71 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:32.275559 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1194/2401] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:58:32.279706 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa6bd1-39f7-4ead-9300-5fedcda2b56c admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:32.290734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa6bd1-39f7-4ead-9300-5fedcda2b56c admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:32.296855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fae9a-1ccf-4aa7-968d-725404c58c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:32.297578 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fae9a-1ccf-4aa7-968d-725404c58c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:32.297856 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fae9a-1ccf-4aa7-968d-725404c58c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:32.298128 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fae9a-1ccf-4aa7-968d-725404c58c0a admin admin] PATH_INFO: `/v3/projects/c68cf8daa1f74af7972c41c17f4720e8/users/45a7d3831bff4619aa3ef2d5dcc3f1f7/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:32.300393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670fae9a-1ccf-4aa7-968d-725404c58c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68cf8daa1f74af7972c41c17f4720e8, user_id=45a7d3831bff4619aa3ef2d5dcc3f1f7, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:32.303079 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670fae9a-1ccf-4aa7-968d-725404c58c0a admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:32.316389 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1208/2402] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:58:32 2022] PUT /identity/v3/projects/c68cf8daa1f74af7972c41c17f4720e8/users/45a7d3831bff4619aa3ef2d5dcc3f1f7/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:32.329059 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad454a4-227b-46fd-a374-fa8fdd671b23 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:32.329656 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad454a4-227b-46fd-a374-fa8fdd671b23 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:32.330073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad454a4-227b-46fd-a374-fa8fdd671b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:32.361150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-5ad454a4-227b-46fd-a374-fa8fdd671b23 None None] MFA Rules not processed for user `45a7d3831bff4619aa3ef2d5dcc3f1f7`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:32.362371 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-5ad454a4-227b-46fd-a374-fa8fdd671b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:32.434802 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1195/2403] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:32 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:58:32.446780 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fae9a-1ccf-4aa7-968d-725404c58c0a admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:32.448574 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-670fae9a-1ccf-4aa7-968d-725404c58c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:32.506947 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fae9a-1ccf-4aa7-968d-725404c58c0a admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:32.510583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8d31b-e470-4f2b-9cda-f96ab772267f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:32.511236 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8d31b-e470-4f2b-9cda-f96ab772267f service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:32.511519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8d31b-e470-4f2b-9cda-f96ab772267f service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:32.511854 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8d31b-e470-4f2b-9cda-f96ab772267f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:32.522491 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8d31b-e470-4f2b-9cda-f96ab772267f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:32.524009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8d31b-e470-4f2b-9cda-f96ab772267f service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:32.534389 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1209/2404] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:58:32 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:58:32.612183 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad454a4-227b-46fd-a374-fa8fdd671b23 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:32.621707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad454a4-227b-46fd-a374-fa8fdd671b23 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:32.627212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbe2c0-d961-4191-8549-91e6f99e7326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:32.627943 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbe2c0-d961-4191-8549-91e6f99e7326 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:32.628246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbe2c0-d961-4191-8549-91e6f99e7326 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:32.628561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbe2c0-d961-4191-8549-91e6f99e7326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:32.642725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:32.689271 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedbe2c0-d961-4191-8549-91e6f99e7326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:32.690547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedbe2c0-d961-4191-8549-91e6f99e7326 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:32.699788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1196/2405] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:58:32 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:58:42.529220 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=36,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:42.695565 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=20 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:47.552442 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8d31b-e470-4f2b-9cda-f96ab772267f service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:47.615234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8d31b-e470-4f2b-9cda-f96ab772267f service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:47.619819 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abcce1-2c85-41fd-8fac-0b58c5183604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:47.620493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abcce1-2c85-41fd-8fac-0b58c5183604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:47.620776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abcce1-2c85-41fd-8fac-0b58c5183604 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:47.621051 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abcce1-2c85-41fd-8fac-0b58c5183604 admin admin] PATH_INFO: `/v3/users/79d818c7df654ec2ac805a6579e905c6` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:47.622223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40abcce1-2c85-41fd-8fac-0b58c5183604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79d818c7df654ec2ac805a6579e905c6)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:47.624186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40abcce1-2c85-41fd-8fac-0b58c5183604 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:47.677723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-40abcce1-2c85-41fd-8fac-0b58c5183604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79d818c7df654ec2ac805a6579e905c6'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:47.686064 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-40abcce1-2c85-41fd-8fac-0b58c5183604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79d818c7df654ec2ac805a6579e905c6'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:47.698948 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-40abcce1-2c85-41fd-8fac-0b58c5183604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79d818c7df654ec2ac805a6579e905c6'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:47.706710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-40abcce1-2c85-41fd-8fac-0b58c5183604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79d818c7df654ec2ac805a6579e905c6'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:47.707450 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-40abcce1-2c85-41fd-8fac-0b58c5183604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79d818c7df654ec2ac805a6579e905c6'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:47.713472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1210/2406] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:58:47 2022] DELETE /identity/v3/users/79d818c7df654ec2ac805a6579e905c6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:47.717881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c9b1a-96e0-41ff-9475-7de820044712 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:47.718351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c9b1a-96e0-41ff-9475-7de820044712 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:47.718463 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c9b1a-96e0-41ff-9475-7de820044712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:47.741177 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-d90c9b1a-96e0-41ff-9475-7de820044712 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:47.742427 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d90c9b1a-96e0-41ff-9475-7de820044712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:47.810298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1197/2407] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:58:47.822104 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abcce1-2c85-41fd-8fac-0b58c5183604 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:47.823727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-40abcce1-2c85-41fd-8fac-0b58c5183604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:47.916410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abcce1-2c85-41fd-8fac-0b58c5183604 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:47.920121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9b933-3b29-4db0-90db-51ada7a88066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:47.920722 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9b933-3b29-4db0-90db-51ada7a88066 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:47.921000 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9b933-3b29-4db0-90db-51ada7a88066 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:47.921273 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9b933-3b29-4db0-90db-51ada7a88066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:47.937061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9b933-3b29-4db0-90db-51ada7a88066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:47.939249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9b933-3b29-4db0-90db-51ada7a88066 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:47.949941 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1211/2408] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:58:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:58:48.400649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c9b1a-96e0-41ff-9475-7de820044712 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:48.404096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d90c9b1a-96e0-41ff-9475-7de820044712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:48.424652 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c9b1a-96e0-41ff-9475-7de820044712 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:48.429869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:48.430510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:48.430810 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:48.431063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a admin admin] PATH_INFO: `/v3/projects/fcd0043f356b4c85bbc4f02f99180aa7` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:48.432214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd0043f356b4c85bbc4f02f99180aa7)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:48.434132 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:48.500032 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd0043f356b4c85bbc4f02f99180aa7', 'request_id': 'req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:48.507734 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd0043f356b4c85bbc4f02f99180aa7', 'request_id': 'req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:48.514494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd0043f356b4c85bbc4f02f99180aa7', 'request_id': 'req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:48.528252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a admin admin] The token cache is being invalidate because project fcd0043f356b4c85bbc4f02f99180aa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:58:48.528581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:48.529252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:48.530750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1198/2409] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:58:48 2022] DELETE /identity/v3/projects/fcd0043f356b4c85bbc4f02f99180aa7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:48.534766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9b933-3b29-4db0-90db-51ada7a88066 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:48.539658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:48.546025 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9b933-3b29-4db0-90db-51ada7a88066 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:48.606863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fc1c4-27d1-4d4e-8c7e-4810e3c10f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:48.608341 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fc1c4-27d1-4d4e-8c7e-4810e3c10f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:48.609404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fc1c4-27d1-4d4e-8c7e-4810e3c10f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:48.609742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fc1c4-27d1-4d4e-8c7e-4810e3c10f78 admin admin] PATH_INFO: `/v3/users/45a7d3831bff4619aa3ef2d5dcc3f1f7` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:48.611617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fc1c4-27d1-4d4e-8c7e-4810e3c10f78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45a7d3831bff4619aa3ef2d5dcc3f1f7)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:48.613488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fc1c4-27d1-4d4e-8c7e-4810e3c10f78 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:48.675622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-597fc1c4-27d1-4d4e-8c7e-4810e3c10f78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45a7d3831bff4619aa3ef2d5dcc3f1f7'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:48.687411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-597fc1c4-27d1-4d4e-8c7e-4810e3c10f78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45a7d3831bff4619aa3ef2d5dcc3f1f7'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:48.711817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-597fc1c4-27d1-4d4e-8c7e-4810e3c10f78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45a7d3831bff4619aa3ef2d5dcc3f1f7'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:48.723240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-597fc1c4-27d1-4d4e-8c7e-4810e3c10f78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45a7d3831bff4619aa3ef2d5dcc3f1f7'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:48.724030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-597fc1c4-27d1-4d4e-8c7e-4810e3c10f78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45a7d3831bff4619aa3ef2d5dcc3f1f7'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:48.733551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1212/2410] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:58:48 2022] DELETE /identity/v3/users/45a7d3831bff4619aa3ef2d5dcc3f1f7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:49.092925 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:49.095538 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:49.112328 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c547c-8ff3-47ab-9152-38e5d0a4060a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:49.198679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fc1c4-27d1-4d4e-8c7e-4810e3c10f78 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:49.200253 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-597fc1c4-27d1-4d4e-8c7e-4810e3c10f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:49.216337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47377b73-f400-4cb9-8e54-a035400cb59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:49.217303 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47377b73-f400-4cb9-8e54-a035400cb59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:49.217746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47377b73-f400-4cb9-8e54-a035400cb59e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:49.218536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47377b73-f400-4cb9-8e54-a035400cb59e admin admin] PATH_INFO: `/v3/projects/c68cf8daa1f74af7972c41c17f4720e8` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:49.222037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47377b73-f400-4cb9-8e54-a035400cb59e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c68cf8daa1f74af7972c41c17f4720e8)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:49.225108 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47377b73-f400-4cb9-8e54-a035400cb59e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:49.271129 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:49.295372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-47377b73-f400-4cb9-8e54-a035400cb59e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c68cf8daa1f74af7972c41c17f4720e8', 'request_id': 'req-47377b73-f400-4cb9-8e54-a035400cb59e'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:49.307300 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-47377b73-f400-4cb9-8e54-a035400cb59e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c68cf8daa1f74af7972c41c17f4720e8', 'request_id': 'req-47377b73-f400-4cb9-8e54-a035400cb59e'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:49.322238 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-47377b73-f400-4cb9-8e54-a035400cb59e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c68cf8daa1f74af7972c41c17f4720e8', 'request_id': 'req-47377b73-f400-4cb9-8e54-a035400cb59e'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:49.343533 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:49.349184 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fc1c4-27d1-4d4e-8c7e-4810e3c10f78 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:49.352098 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-47377b73-f400-4cb9-8e54-a035400cb59e admin admin] The token cache is being invalidate because project c68cf8daa1f74af7972c41c17f4720e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:58:49.352827 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-47377b73-f400-4cb9-8e54-a035400cb59e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:49.354418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-47377b73-f400-4cb9-8e54-a035400cb59e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:49.356605 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1199/2411] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:58:49 2022] DELETE /identity/v3/projects/c68cf8daa1f74af7972c41c17f4720e8 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:58:49.363538 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375b482-11fe-4172-8b0b-a14cdfed15c6 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:49.363796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375b482-11fe-4172-8b0b-a14cdfed15c6 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:49.363970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375b482-11fe-4172-8b0b-a14cdfed15c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:49.388862 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-8375b482-11fe-4172-8b0b-a14cdfed15c6 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:49.390853 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8375b482-11fe-4172-8b0b-a14cdfed15c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:49.463419 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c653f-e26b-4c6c-b45b-da10e550909c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:49.463701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c653f-e26b-4c6c-b45b-da10e550909c service placement] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:49.463897 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c653f-e26b-4c6c-b45b-da10e550909c service placement] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:49.464179 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c653f-e26b-4c6c-b45b-da10e550909c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:49.466171 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-3b9c653f-e26b-4c6c-b45b-da10e550909c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:49.467560 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1200/2412] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:58:49.476539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee137623-182c-474a-a9a2-1bb65e101cc1 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:49.476868 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee137623-182c-474a-a9a2-1bb65e101cc1 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:49.477073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee137623-182c-474a-a9a2-1bb65e101cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:49.493252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c653f-e26b-4c6c-b45b-da10e550909c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:49.495997 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c653f-e26b-4c6c-b45b-da10e550909c service placement] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:49.518602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ee137623-182c-474a-a9a2-1bb65e101cc1 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:49.520900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ee137623-182c-474a-a9a2-1bb65e101cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:49.521935 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1213/2413] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:58:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:58:49.536624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1201/2414] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:58:49.537701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c653f-e26b-4c6c-b45b-da10e550909c service placement] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:49.551763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c653f-e26b-4c6c-b45b-da10e550909c service placement] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:49.556489 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b475419-3806-434c-9b69-4d76e37e6515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:49.557081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b475419-3806-434c-9b69-4d76e37e6515 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:49.557363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b475419-3806-434c-9b69-4d76e37e6515 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:49.557637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b475419-3806-434c-9b69-4d76e37e6515 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:49.558342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b475419-3806-434c-9b69-4d76e37e6515 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:58:49.559190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b475419-3806-434c-9b69-4d76e37e6515 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:49.567501 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b475419-3806-434c-9b69-4d76e37e6515 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:49.583784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1214/2415] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:58:49.590237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee137623-182c-474a-a9a2-1bb65e101cc1 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:49.603335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee137623-182c-474a-a9a2-1bb65e101cc1 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:49.611397 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed68b9f-721d-4bf3-9f2e-82274d12aa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:49.612044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed68b9f-721d-4bf3-9f2e-82274d12aa87 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:49.612438 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed68b9f-721d-4bf3-9f2e-82274d12aa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:49.612725 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed68b9f-721d-4bf3-9f2e-82274d12aa87 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:49.613457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed68b9f-721d-4bf3-9f2e-82274d12aa87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:49.616436 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed68b9f-721d-4bf3-9f2e-82274d12aa87 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:49.683324 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1202/2416] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:58:49 2022] POST /identity/v3/projects => generated 367 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:58:49.687562 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b475419-3806-434c-9b69-4d76e37e6515 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:49.700791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b475419-3806-434c-9b69-4d76e37e6515 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:49.800305 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a27e2-f254-4498-a3cc-caffe8b4df9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:49.801520 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a27e2-f254-4498-a3cc-caffe8b4df9c admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:49.801939 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a27e2-f254-4498-a3cc-caffe8b4df9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:49.802310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a27e2-f254-4498-a3cc-caffe8b4df9c admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:49.803094 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a27e2-f254-4498-a3cc-caffe8b4df9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:49.806370 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a27e2-f254-4498-a3cc-caffe8b4df9c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:49.856037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1215/2417] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:58:49 2022] POST /identity/v3/users => generated 284 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:58:49.860684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed68b9f-721d-4bf3-9f2e-82274d12aa87 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:49.873148 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed68b9f-721d-4bf3-9f2e-82274d12aa87 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:49.881767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd7989-5db7-4cd2-9d64-9226f3b6eb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:49.882493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd7989-5db7-4cd2-9d64-9226f3b6eb0a admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:49.882834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd7989-5db7-4cd2-9d64-9226f3b6eb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:49.883170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd7989-5db7-4cd2-9d64-9226f3b6eb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:49.883980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd7989-5db7-4cd2-9d64-9226f3b6eb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:49.886113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd7989-5db7-4cd2-9d64-9226f3b6eb0a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:49.905489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1203/2418] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:58:49.910688 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a27e2-f254-4498-a3cc-caffe8b4df9c admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:49.936747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a27e2-f254-4498-a3cc-caffe8b4df9c admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:49.954233 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ab8ba-e8cb-41e5-87c2-a2f725ddc51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:49.958679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ab8ba-e8cb-41e5-87c2-a2f725ddc51a admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:49.958833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ab8ba-e8cb-41e5-87c2-a2f725ddc51a admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:49.959059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ab8ba-e8cb-41e5-87c2-a2f725ddc51a admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:49.961659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ab8ba-e8cb-41e5-87c2-a2f725ddc51a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:49.965124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ab8ba-e8cb-41e5-87c2-a2f725ddc51a admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:49.988695 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1216/2419] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:58:49.990402 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd7989-5db7-4cd2-9d64-9226f3b6eb0a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:50.003304 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd7989-5db7-4cd2-9d64-9226f3b6eb0a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:50.009280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ea15d-c9c7-47ff-a34e-25d322f9fd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:50.010253 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ea15d-c9c7-47ff-a34e-25d322f9fd00 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:50.010805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ea15d-c9c7-47ff-a34e-25d322f9fd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:50.011274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ea15d-c9c7-47ff-a34e-25d322f9fd00 admin admin] PATH_INFO: `/v3/projects/c48054a1757f4a6792582f8ba14bc2a5/users/cfd304852147405b93f7a873d3403b98/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:50.037592 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ea15d-c9c7-47ff-a34e-25d322f9fd00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c48054a1757f4a6792582f8ba14bc2a5, user_id=cfd304852147405b93f7a873d3403b98, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:50.044016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ea15d-c9c7-47ff-a34e-25d322f9fd00 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:50.080975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1204/2420] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:58:49 2022] PUT /identity/v3/projects/c48054a1757f4a6792582f8ba14bc2a5/users/cfd304852147405b93f7a873d3403b98/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:50.082933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26546c9f-a509-4c28-80fe-b7d6fa272453 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:50.083447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26546c9f-a509-4c28-80fe-b7d6fa272453 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:50.083921 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26546c9f-a509-4c28-80fe-b7d6fa272453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:50.116756 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-26546c9f-a509-4c28-80fe-b7d6fa272453 None None] MFA Rules not processed for user `cfd304852147405b93f7a873d3403b98`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:50.118393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-26546c9f-a509-4c28-80fe-b7d6fa272453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:50.188525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:50.208308 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1217/2421] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:58:50.212830 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ea15d-c9c7-47ff-a34e-25d322f9fd00 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:50.266748 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:50.279945 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ea15d-c9c7-47ff-a34e-25d322f9fd00 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:50.286589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ab6a6-1a2f-4ddc-a2f0-fbded400be85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:50.287147 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ab6a6-1a2f-4ddc-a2f0-fbded400be85 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:50.287435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ab6a6-1a2f-4ddc-a2f0-fbded400be85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:50.287716 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ab6a6-1a2f-4ddc-a2f0-fbded400be85 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:50.288507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ab6a6-1a2f-4ddc-a2f0-fbded400be85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:50.290435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ab6a6-1a2f-4ddc-a2f0-fbded400be85 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:50.322211 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1205/2422] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:58:50 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:58:50.323206 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26546c9f-a509-4c28-80fe-b7d6fa272453 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:50.336345 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26546c9f-a509-4c28-80fe-b7d6fa272453 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:50.401760 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709c916-d21c-4b12-a641-d4ce73c7b8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:50.402493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709c916-d21c-4b12-a641-d4ce73c7b8e6 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:50.402831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709c916-d21c-4b12-a641-d4ce73c7b8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:50.403140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709c916-d21c-4b12-a641-d4ce73c7b8e6 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:50.403841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709c916-d21c-4b12-a641-d4ce73c7b8e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:50.405848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709c916-d21c-4b12-a641-d4ce73c7b8e6 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:50.429340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1218/2423] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:58:50 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 18:58:50.433238 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ab6a6-1a2f-4ddc-a2f0-fbded400be85 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:50.444147 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ab6a6-1a2f-4ddc-a2f0-fbded400be85 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:50.449282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef8646-a754-4819-af0e-479179c22f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:50.450322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef8646-a754-4819-af0e-479179c22f80 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:50.450322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef8646-a754-4819-af0e-479179c22f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:50.450552 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef8646-a754-4819-af0e-479179c22f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:50.451238 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef8646-a754-4819-af0e-479179c22f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:50.452981 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef8646-a754-4819-af0e-479179c22f80 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:50.464723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1206/2424] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:58:50.468853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709c916-d21c-4b12-a641-d4ce73c7b8e6 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:50.480159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709c916-d21c-4b12-a641-d4ce73c7b8e6 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:50.485059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0422476-eeb9-49d6-8e0c-cf26d493865b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:50.485741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0422476-eeb9-49d6-8e0c-cf26d493865b admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:50.486290 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0422476-eeb9-49d6-8e0c-cf26d493865b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:50.486290 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0422476-eeb9-49d6-8e0c-cf26d493865b admin admin] PATH_INFO: `/v3/projects/3973cdcf1fff4eb58aac389ffa338c5f/users/b65c3fd7cff242f49170e1460faeee14/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:50.511677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0422476-eeb9-49d6-8e0c-cf26d493865b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3973cdcf1fff4eb58aac389ffa338c5f, user_id=b65c3fd7cff242f49170e1460faeee14, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:50.515078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0422476-eeb9-49d6-8e0c-cf26d493865b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:50.530767 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1219/2425] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:58:50 2022] PUT /identity/v3/projects/3973cdcf1fff4eb58aac389ffa338c5f/users/b65c3fd7cff242f49170e1460faeee14/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:50.535959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef8646-a754-4819-af0e-479179c22f80 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:50.546128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef8646-a754-4819-af0e-479179c22f80 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:50.605733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd4093-9656-4c6f-ae85-b0172cc5f44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:50.606362 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd4093-9656-4c6f-ae85-b0172cc5f44a admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:50.606647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd4093-9656-4c6f-ae85-b0172cc5f44a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:50.606923 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd4093-9656-4c6f-ae85-b0172cc5f44a admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:50.607618 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd4093-9656-4c6f-ae85-b0172cc5f44a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:50.609263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd4093-9656-4c6f-ae85-b0172cc5f44a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:50.620652 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1207/2426] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:58:50.625107 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0422476-eeb9-49d6-8e0c-cf26d493865b admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:50.636822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0422476-eeb9-49d6-8e0c-cf26d493865b admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:50.642179 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea8bf1-fb86-4307-a522-7c90e2224771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:50.642859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea8bf1-fb86-4307-a522-7c90e2224771 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:50.643141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea8bf1-fb86-4307-a522-7c90e2224771 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:50.643417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea8bf1-fb86-4307-a522-7c90e2224771 admin admin] PATH_INFO: `/v3/projects/3973cdcf1fff4eb58aac389ffa338c5f/users/b65c3fd7cff242f49170e1460faeee14/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:50.645784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea8bf1-fb86-4307-a522-7c90e2224771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3973cdcf1fff4eb58aac389ffa338c5f, user_id=b65c3fd7cff242f49170e1460faeee14, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:50.648476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea8bf1-fb86-4307-a522-7c90e2224771 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:50.659089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1220/2427] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:58:50 2022] PUT /identity/v3/projects/3973cdcf1fff4eb58aac389ffa338c5f/users/b65c3fd7cff242f49170e1460faeee14/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:50.669217 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1978904-5a1f-4e8c-9728-d6a42c3aa8bc None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:50.669501 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1978904-5a1f-4e8c-9728-d6a42c3aa8bc None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:50.669771 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1978904-5a1f-4e8c-9728-d6a42c3aa8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:50.700542 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-b1978904-5a1f-4e8c-9728-d6a42c3aa8bc None None] MFA Rules not processed for user `b65c3fd7cff242f49170e1460faeee14`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:50.701879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b1978904-5a1f-4e8c-9728-d6a42c3aa8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:50.794351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1208/2428] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:58:50.806791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea8bf1-fb86-4307-a522-7c90e2224771 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:50.809312 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-05ea8bf1-fb86-4307-a522-7c90e2224771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:50.897798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea8bf1-fb86-4307-a522-7c90e2224771 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:50.901471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53257467-d4a2-4bc5-bce9-4bf4357bea74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:50.902088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53257467-d4a2-4bc5-bce9-4bf4357bea74 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:50.902368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53257467-d4a2-4bc5-bce9-4bf4357bea74 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:50.902639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53257467-d4a2-4bc5-bce9-4bf4357bea74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:50.913344 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53257467-d4a2-4bc5-bce9-4bf4357bea74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:50.915040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53257467-d4a2-4bc5-bce9-4bf4357bea74 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:50.926130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1221/2429] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:58:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:58:52.807682 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1978904-5a1f-4e8c-9728-d6a42c3aa8bc None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:52.884901 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:52.918229 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1978904-5a1f-4e8c-9728-d6a42c3aa8bc None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:52.929994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60aafab-28df-4a54-9746-9879993968f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:52.930663 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aafab-28df-4a54-9746-9879993968f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:52.931091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aafab-28df-4a54-9746-9879993968f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:52.931373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aafab-28df-4a54-9746-9879993968f8 admin admin] PATH_INFO: `/v3/users/cfd304852147405b93f7a873d3403b98` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:52.932628 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60aafab-28df-4a54-9746-9879993968f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfd304852147405b93f7a873d3403b98)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:52.934584 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60aafab-28df-4a54-9746-9879993968f8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:53.001931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e60aafab-28df-4a54-9746-9879993968f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfd304852147405b93f7a873d3403b98'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:53.013125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e60aafab-28df-4a54-9746-9879993968f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfd304852147405b93f7a873d3403b98'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:53.033212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e60aafab-28df-4a54-9746-9879993968f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfd304852147405b93f7a873d3403b98'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:53.041840 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e60aafab-28df-4a54-9746-9879993968f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfd304852147405b93f7a873d3403b98'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:53.042245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e60aafab-28df-4a54-9746-9879993968f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfd304852147405b93f7a873d3403b98'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:53.048469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1209/2430] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:58:52 2022] DELETE /identity/v3/users/cfd304852147405b93f7a873d3403b98 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:58:53.053688 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc8dc5-0fb2-454c-98df-07f54d60ebad None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:53.054143 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc8dc5-0fb2-454c-98df-07f54d60ebad None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:53.054593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc8dc5-0fb2-454c-98df-07f54d60ebad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:53.079019 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-67bc8dc5-0fb2-454c-98df-07f54d60ebad None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:53.082343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-67bc8dc5-0fb2-454c-98df-07f54d60ebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:53.101647 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:53.158174 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1222/2431] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:58:53.169324 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60aafab-28df-4a54-9746-9879993968f8 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:53.171293 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e60aafab-28df-4a54-9746-9879993968f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:53.242898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60aafab-28df-4a54-9746-9879993968f8 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:53.246666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62db378-5f23-49eb-8fc8-2e94f45d90e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:53.247365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62db378-5f23-49eb-8fc8-2e94f45d90e0 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:53.247739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62db378-5f23-49eb-8fc8-2e94f45d90e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:53.248080 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62db378-5f23-49eb-8fc8-2e94f45d90e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:53.258238 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62db378-5f23-49eb-8fc8-2e94f45d90e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:53.259774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62db378-5f23-49eb-8fc8-2e94f45d90e0 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:53.270177 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1210/2432] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:58:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:58:53.563990 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc8dc5-0fb2-454c-98df-07f54d60ebad None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:53.565755 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-67bc8dc5-0fb2-454c-98df-07f54d60ebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:53.589780 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc8dc5-0fb2-454c-98df-07f54d60ebad None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:53.600685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6de82e-6ac9-41d2-8bf6-566586601b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:53.601461 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6de82e-6ac9-41d2-8bf6-566586601b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:53.601750 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6de82e-6ac9-41d2-8bf6-566586601b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:53.602033 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6de82e-6ac9-41d2-8bf6-566586601b49 admin admin] PATH_INFO: `/v3/projects/c48054a1757f4a6792582f8ba14bc2a5` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:53.603247 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6de82e-6ac9-41d2-8bf6-566586601b49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c48054a1757f4a6792582f8ba14bc2a5)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:53.605243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6de82e-6ac9-41d2-8bf6-566586601b49 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:53.659932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5b6de82e-6ac9-41d2-8bf6-566586601b49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c48054a1757f4a6792582f8ba14bc2a5', 'request_id': 'req-5b6de82e-6ac9-41d2-8bf6-566586601b49'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:53.665751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5b6de82e-6ac9-41d2-8bf6-566586601b49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c48054a1757f4a6792582f8ba14bc2a5', 'request_id': 'req-5b6de82e-6ac9-41d2-8bf6-566586601b49'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:53.675077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5b6de82e-6ac9-41d2-8bf6-566586601b49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c48054a1757f4a6792582f8ba14bc2a5', 'request_id': 'req-5b6de82e-6ac9-41d2-8bf6-566586601b49'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:53.687104 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5b6de82e-6ac9-41d2-8bf6-566586601b49 admin admin] The token cache is being invalidate because project c48054a1757f4a6792582f8ba14bc2a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:58:53.687907 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5b6de82e-6ac9-41d2-8bf6-566586601b49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:53.689136 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5b6de82e-6ac9-41d2-8bf6-566586601b49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:53.690924 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1223/2433] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:58:53 2022] DELETE /identity/v3/projects/c48054a1757f4a6792582f8ba14bc2a5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:53.694962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62db378-5f23-49eb-8fc8-2e94f45d90e0 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:53.705662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62db378-5f23-49eb-8fc8-2e94f45d90e0 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:53.765132 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c21ffd-4517-40d4-8534-e5e36bbd30a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:53.766982 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c21ffd-4517-40d4-8534-e5e36bbd30a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:53.766982 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c21ffd-4517-40d4-8534-e5e36bbd30a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:53.766982 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c21ffd-4517-40d4-8534-e5e36bbd30a4 admin admin] PATH_INFO: `/v3/users/b65c3fd7cff242f49170e1460faeee14` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:53.768494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c21ffd-4517-40d4-8534-e5e36bbd30a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b65c3fd7cff242f49170e1460faeee14)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:53.770364 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c21ffd-4517-40d4-8534-e5e36bbd30a4 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:53.782763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:53.825093 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-60c21ffd-4517-40d4-8534-e5e36bbd30a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b65c3fd7cff242f49170e1460faeee14'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:53.834191 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-60c21ffd-4517-40d4-8534-e5e36bbd30a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b65c3fd7cff242f49170e1460faeee14'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:53.847880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-60c21ffd-4517-40d4-8534-e5e36bbd30a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b65c3fd7cff242f49170e1460faeee14'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:53.855835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-60c21ffd-4517-40d4-8534-e5e36bbd30a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b65c3fd7cff242f49170e1460faeee14'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:53.856592 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-60c21ffd-4517-40d4-8534-e5e36bbd30a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b65c3fd7cff242f49170e1460faeee14'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:53.863278 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1211/2434] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:58:53 2022] DELETE /identity/v3/users/b65c3fd7cff242f49170e1460faeee14 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:54.050119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79959fc5-ba89-427d-8462-6f66f2da93b1 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:54.050405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79959fc5-ba89-427d-8462-6f66f2da93b1 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:54.050685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79959fc5-ba89-427d-8462-6f66f2da93b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:54.071085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-79959fc5-ba89-427d-8462-6f66f2da93b1 None None] MFA Rules not processed for user `13ab977f852a41e898af1ddfb67bad69`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:54.072317 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-79959fc5-ba89-427d-8462-6f66f2da93b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:54.130416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1224/2435] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:58:54.163625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c21ffd-4517-40d4-8534-e5e36bbd30a4 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:54.165247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-60c21ffd-4517-40d4-8534-e5e36bbd30a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:54.248108 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c21ffd-4517-40d4-8534-e5e36bbd30a4 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:54.254650 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2aa45-c2ab-4dc8-9cc3-79141cefcdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:54.255303 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2aa45-c2ab-4dc8-9cc3-79141cefcdd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:54.255649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2aa45-c2ab-4dc8-9cc3-79141cefcdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:54.255844 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2aa45-c2ab-4dc8-9cc3-79141cefcdd7 admin admin] PATH_INFO: `/v3/projects/3973cdcf1fff4eb58aac389ffa338c5f` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:54.257153 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2aa45-c2ab-4dc8-9cc3-79141cefcdd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3973cdcf1fff4eb58aac389ffa338c5f)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:54.259298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2aa45-c2ab-4dc8-9cc3-79141cefcdd7 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:54.317365 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e2e2aa45-c2ab-4dc8-9cc3-79141cefcdd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3973cdcf1fff4eb58aac389ffa338c5f', 'request_id': 'req-e2e2aa45-c2ab-4dc8-9cc3-79141cefcdd7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:54.324059 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e2e2aa45-c2ab-4dc8-9cc3-79141cefcdd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3973cdcf1fff4eb58aac389ffa338c5f', 'request_id': 'req-e2e2aa45-c2ab-4dc8-9cc3-79141cefcdd7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:54.331113 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e2e2aa45-c2ab-4dc8-9cc3-79141cefcdd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3973cdcf1fff4eb58aac389ffa338c5f', 'request_id': 'req-e2e2aa45-c2ab-4dc8-9cc3-79141cefcdd7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:54.348368 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e2e2aa45-c2ab-4dc8-9cc3-79141cefcdd7 admin admin] The token cache is being invalidate because project 3973cdcf1fff4eb58aac389ffa338c5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:58:54.348909 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e2e2aa45-c2ab-4dc8-9cc3-79141cefcdd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:54.349901 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e2e2aa45-c2ab-4dc8-9cc3-79141cefcdd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:58:54.351802 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1212/2436] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:58:54 2022] DELETE /identity/v3/projects/3973cdcf1fff4eb58aac389ffa338c5f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:54.356596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87715b-8732-4f0b-8dff-2b7458c5b96a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:54.357807 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87715b-8732-4f0b-8dff-2b7458c5b96a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:54.358369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87715b-8732-4f0b-8dff-2b7458c5b96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:54.381435 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-dd87715b-8732-4f0b-8dff-2b7458c5b96a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:54.382794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-dd87715b-8732-4f0b-8dff-2b7458c5b96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:54.435766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:54.450789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1225/2437] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 18:58:54.459075 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0345d2-f0b5-4363-b7c7-861b85c2a1b7 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:54.459367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0345d2-f0b5-4363-b7c7-861b85c2a1b7 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:54.459674 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0345d2-f0b5-4363-b7c7-861b85c2a1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:54.480183 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ea0345d2-f0b5-4363-b7c7-861b85c2a1b7 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:54.481560 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ea0345d2-f0b5-4363-b7c7-861b85c2a1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:54.490547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1213/2438] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:58:54.494665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87715b-8732-4f0b-8dff-2b7458c5b96a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:54.508398 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87715b-8732-4f0b-8dff-2b7458c5b96a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:54.514007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c6f0f-35f7-4c1c-bde1-f6bf0b5804ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:54.514644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c6f0f-35f7-4c1c-bde1-f6bf0b5804ed admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:54.514926 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c6f0f-35f7-4c1c-bde1-f6bf0b5804ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:54.515198 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c6f0f-35f7-4c1c-bde1-f6bf0b5804ed admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:54.516278 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c6f0f-35f7-4c1c-bde1-f6bf0b5804ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:58:54.516700 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c6f0f-35f7-4c1c-bde1-f6bf0b5804ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:54.518315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c6f0f-35f7-4c1c-bde1-f6bf0b5804ed admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:54.532687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1226/2439] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:58:54.535652 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0345d2-f0b5-4363-b7c7-861b85c2a1b7 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:54.544906 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0345d2-f0b5-4363-b7c7-861b85c2a1b7 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:54.550476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6cc29c-990c-474b-8a68-b2564d6ffac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:54.551098 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cc29c-990c-474b-8a68-b2564d6ffac2 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:54.551380 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cc29c-990c-474b-8a68-b2564d6ffac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:54.551653 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cc29c-990c-474b-8a68-b2564d6ffac2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:54.552381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6cc29c-990c-474b-8a68-b2564d6ffac2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:54.554295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6cc29c-990c-474b-8a68-b2564d6ffac2 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:54.565254 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:54.589136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1214/2440] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:58:54 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:58:54.590296 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c6f0f-35f7-4c1c-bde1-f6bf0b5804ed admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:54.601346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c6f0f-35f7-4c1c-bde1-f6bf0b5804ed admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:54.663626 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bea51-da30-4f06-aa67-49027a353b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:54.664278 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bea51-da30-4f06-aa67-49027a353b22 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:54.664581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bea51-da30-4f06-aa67-49027a353b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:54.664849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bea51-da30-4f06-aa67-49027a353b22 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:54.665508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bea51-da30-4f06-aa67-49027a353b22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:54.667416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bea51-da30-4f06-aa67-49027a353b22 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:54.694561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1227/2441] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:58:54 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:58:54.695757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6cc29c-990c-474b-8a68-b2564d6ffac2 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:54.707239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6cc29c-990c-474b-8a68-b2564d6ffac2 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:54.711914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38681f4-eddb-4f0e-b8ea-b1d2e4c29b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:54.712566 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38681f4-eddb-4f0e-b8ea-b1d2e4c29b76 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:54.712846 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38681f4-eddb-4f0e-b8ea-b1d2e4c29b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:54.713120 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38681f4-eddb-4f0e-b8ea-b1d2e4c29b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:54.713852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38681f4-eddb-4f0e-b8ea-b1d2e4c29b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:54.715489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38681f4-eddb-4f0e-b8ea-b1d2e4c29b76 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:54.726522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1215/2442] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:58:54.730746 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bea51-da30-4f06-aa67-49027a353b22 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:54.741066 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bea51-da30-4f06-aa67-49027a353b22 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:54.745689 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8113e7-47a9-4048-8cba-adac4f2183c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:54.746301 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8113e7-47a9-4048-8cba-adac4f2183c4 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:54.746585 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8113e7-47a9-4048-8cba-adac4f2183c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:54.746861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8113e7-47a9-4048-8cba-adac4f2183c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:54.747589 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8113e7-47a9-4048-8cba-adac4f2183c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:54.749248 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8113e7-47a9-4048-8cba-adac4f2183c4 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:54.760890 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1228/2443] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:58:54.765117 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38681f4-eddb-4f0e-b8ea-b1d2e4c29b76 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:54.779882 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38681f4-eddb-4f0e-b8ea-b1d2e4c29b76 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:54.787289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe8697-5319-4e61-9c02-2cdfbd6cf020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:54.788414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe8697-5319-4e61-9c02-2cdfbd6cf020 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:54.788935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe8697-5319-4e61-9c02-2cdfbd6cf020 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:54.789416 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe8697-5319-4e61-9c02-2cdfbd6cf020 admin admin] PATH_INFO: `/v3/projects/d0d1ec9f5bb14df6a0335c2993c065f8/users/6880fdaab78545af91954a8df86ee06a/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:54.813910 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe8697-5319-4e61-9c02-2cdfbd6cf020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0d1ec9f5bb14df6a0335c2993c065f8, user_id=6880fdaab78545af91954a8df86ee06a, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:54.819019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe8697-5319-4e61-9c02-2cdfbd6cf020 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:54.835308 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1216/2444] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:58:54 2022] PUT /identity/v3/projects/d0d1ec9f5bb14df6a0335c2993c065f8/users/6880fdaab78545af91954a8df86ee06a/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:54.846985 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ce5e1-f684-4ec6-ba9d-4eac5fd1eba6 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:54.847274 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ce5e1-f684-4ec6-ba9d-4eac5fd1eba6 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:54.847551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ce5e1-f684-4ec6-ba9d-4eac5fd1eba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:54.877406 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-ee2ce5e1-f684-4ec6-ba9d-4eac5fd1eba6 None None] MFA Rules not processed for user `6880fdaab78545af91954a8df86ee06a`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:54.878914 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ee2ce5e1-f684-4ec6-ba9d-4eac5fd1eba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:54.953981 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1229/2445] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:58:54.958594 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe8697-5319-4e61-9c02-2cdfbd6cf020 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:55.027757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe8697-5319-4e61-9c02-2cdfbd6cf020 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:55.032560 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2b786-64be-4429-bb07-d37faaa2878c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:55.033304 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2b786-64be-4429-bb07-d37faaa2878c admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:55.033457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2b786-64be-4429-bb07-d37faaa2878c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:55.033732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2b786-64be-4429-bb07-d37faaa2878c admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:55.034441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2b786-64be-4429-bb07-d37faaa2878c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:55.036376 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2b786-64be-4429-bb07-d37faaa2878c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:55.064162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1217/2446] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:58:54 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:58:55.068076 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ce5e1-f684-4ec6-ba9d-4eac5fd1eba6 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:55.079739 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ce5e1-f684-4ec6-ba9d-4eac5fd1eba6 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:55.127752 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:55.145754 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831e45c-a13b-4e54-9df8-f1a9b442cf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:55.146398 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e45c-a13b-4e54-9df8-f1a9b442cf41 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:55.146683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e45c-a13b-4e54-9df8-f1a9b442cf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:55.146958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e45c-a13b-4e54-9df8-f1a9b442cf41 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:55.147629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831e45c-a13b-4e54-9df8-f1a9b442cf41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:55.149792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831e45c-a13b-4e54-9df8-f1a9b442cf41 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:55.171524 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1230/2447] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:58:55 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:58:55.176852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2b786-64be-4429-bb07-d37faaa2878c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:55.189190 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2b786-64be-4429-bb07-d37faaa2878c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:55.194337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6419a0-fa21-4a76-bd19-e6aa6f62660e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:55.194967 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6419a0-fa21-4a76-bd19-e6aa6f62660e admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:55.195251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6419a0-fa21-4a76-bd19-e6aa6f62660e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:55.195526 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6419a0-fa21-4a76-bd19-e6aa6f62660e admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:55.196232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6419a0-fa21-4a76-bd19-e6aa6f62660e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:55.197865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6419a0-fa21-4a76-bd19-e6aa6f62660e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:55.212687 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1218/2448] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:58:55.214784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831e45c-a13b-4e54-9df8-f1a9b442cf41 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:55.225238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831e45c-a13b-4e54-9df8-f1a9b442cf41 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:55.230261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7592f7-7c21-49da-8850-fa760adf385b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:55.230940 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7592f7-7c21-49da-8850-fa760adf385b admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:55.231223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7592f7-7c21-49da-8850-fa760adf385b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:55.231501 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7592f7-7c21-49da-8850-fa760adf385b admin admin] PATH_INFO: `/v3/projects/5dbea53f2b4c484cb96fcf350cd366b8/users/92aba7679ade4116a9095ac54e000ae4/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:55.248134 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7592f7-7c21-49da-8850-fa760adf385b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dbea53f2b4c484cb96fcf350cd366b8, user_id=92aba7679ade4116a9095ac54e000ae4, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:55.252098 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7592f7-7c21-49da-8850-fa760adf385b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:55.266358 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1231/2449] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:58:55 2022] PUT /identity/v3/projects/5dbea53f2b4c484cb96fcf350cd366b8/users/92aba7679ade4116a9095ac54e000ae4/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:55.270593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6419a0-fa21-4a76-bd19-e6aa6f62660e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:55.282422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6419a0-fa21-4a76-bd19-e6aa6f62660e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:55.339790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbd8a2-9d08-4607-a48a-f8496334e4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:55.340454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbd8a2-9d08-4607-a48a-f8496334e4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:55.340741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbd8a2-9d08-4607-a48a-f8496334e4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:55.341018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbd8a2-9d08-4607-a48a-f8496334e4a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:55.341707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbd8a2-9d08-4607-a48a-f8496334e4a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:55.343309 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbd8a2-9d08-4607-a48a-f8496334e4a8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:55.353284 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1219/2450] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:58:55.357487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7592f7-7c21-49da-8850-fa760adf385b admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:55.368072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7592f7-7c21-49da-8850-fa760adf385b admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:55.372626 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50520c30-3df1-476c-a682-b26e0589d7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:55.373279 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50520c30-3df1-476c-a682-b26e0589d7e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:55.373561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50520c30-3df1-476c-a682-b26e0589d7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:55.373836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50520c30-3df1-476c-a682-b26e0589d7e2 admin admin] PATH_INFO: `/v3/projects/5dbea53f2b4c484cb96fcf350cd366b8/users/92aba7679ade4116a9095ac54e000ae4/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:55.376785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50520c30-3df1-476c-a682-b26e0589d7e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dbea53f2b4c484cb96fcf350cd366b8, user_id=92aba7679ade4116a9095ac54e000ae4, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:55.379498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50520c30-3df1-476c-a682-b26e0589d7e2 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:55.389883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1232/2451] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:58:55 2022] PUT /identity/v3/projects/5dbea53f2b4c484cb96fcf350cd366b8/users/92aba7679ade4116a9095ac54e000ae4/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:58:55.400514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc76256-c8cd-437e-acd9-5038b265cbf6 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:55.400886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc76256-c8cd-437e-acd9-5038b265cbf6 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:55.401419 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc76256-c8cd-437e-acd9-5038b265cbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:55.431203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-dcc76256-c8cd-437e-acd9-5038b265cbf6 None None] MFA Rules not processed for user `92aba7679ade4116a9095ac54e000ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:55.432480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-dcc76256-c8cd-437e-acd9-5038b265cbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:55.444998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:55.504075 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1220/2452] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:58:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:58:55.514868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50520c30-3df1-476c-a682-b26e0589d7e2 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:55.516937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-50520c30-3df1-476c-a682-b26e0589d7e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:55.575193 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50520c30-3df1-476c-a682-b26e0589d7e2 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:55.579454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb58d7-bf75-47dc-8e5b-f7c392b359c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:55.580069 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb58d7-bf75-47dc-8e5b-f7c392b359c5 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:55.580391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb58d7-bf75-47dc-8e5b-f7c392b359c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:55.580659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb58d7-bf75-47dc-8e5b-f7c392b359c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:55.638309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb58d7-bf75-47dc-8e5b-f7c392b359c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:55.639674 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb58d7-bf75-47dc-8e5b-f7c392b359c5 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:55.648829 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1233/2453] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:58:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:58:59.403489 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969c324-5ff0-4667-803a-70551cf83b9f None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:59.403966 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969c324-5ff0-4667-803a-70551cf83b9f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:59.404085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969c324-5ff0-4667-803a-70551cf83b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:59.435314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-1969c324-5ff0-4667-803a-70551cf83b9f None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:58:59.436584 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1969c324-5ff0-4667-803a-70551cf83b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:59.498108 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1221/2454] 10.222.0.69 () {58 vars in 967 bytes} [Wed Aug 3 18:58:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:58:59.553221 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb58d7-bf75-47dc-8e5b-f7c392b359c5 service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:58:59.554633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4abb58d7-bf75-47dc-8e5b-f7c392b359c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:58:59.612343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:58:59.628627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb58d7-bf75-47dc-8e5b-f7c392b359c5 service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:58:59.633775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43858806-8996-441e-ab19-42d2e40db664 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:58:59.634410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43858806-8996-441e-ab19-42d2e40db664 service swift] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:58:59.634701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43858806-8996-441e-ab19-42d2e40db664 service swift] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:58:59.634976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43858806-8996-441e-ab19-42d2e40db664 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:58:59.643876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43858806-8996-441e-ab19-42d2e40db664 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:58:59.645500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43858806-8996-441e-ab19-42d2e40db664 service swift] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:58:59.653529 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1234/2455] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 18:58:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 18:59:00.016586 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969c324-5ff0-4667-803a-70551cf83b9f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:00.018734 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1969c324-5ff0-4667-803a-70551cf83b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:00.030128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969c324-5ff0-4667-803a-70551cf83b9f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:00.033866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c135ca6-69dc-44c8-9dad-603eaf90ee00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:00.034551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c135ca6-69dc-44c8-9dad-603eaf90ee00 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:00.034947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c135ca6-69dc-44c8-9dad-603eaf90ee00 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:00.035274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c135ca6-69dc-44c8-9dad-603eaf90ee00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:00.045263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c135ca6-69dc-44c8-9dad-603eaf90ee00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:00.046787 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c135ca6-69dc-44c8-9dad-603eaf90ee00 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:00.056996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1222/2456] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:59:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:59:03.544967 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43858806-8996-441e-ab19-42d2e40db664 service swift] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:03.546470 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-43858806-8996-441e-ab19-42d2e40db664 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:03.634357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43858806-8996-441e-ab19-42d2e40db664 service swift] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:03.640275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b5223-7636-407b-bc4c-093a8cd71bd9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:03.641030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b5223-7636-407b-bc4c-093a8cd71bd9 service placement] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:03.641433 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b5223-7636-407b-bc4c-093a8cd71bd9 service placement] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:03.641753 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b5223-7636-407b-bc4c-093a8cd71bd9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:03.644717 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7f3b5223-7636-407b-bc4c-093a8cd71bd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:03.657973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b5223-7636-407b-bc4c-093a8cd71bd9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:03.660124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b5223-7636-407b-bc4c-093a8cd71bd9 service placement] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:03.670638 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1235/2457] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:59:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:59:05.025733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c135ca6-69dc-44c8-9dad-603eaf90ee00 service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:05.097922 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c135ca6-69dc-44c8-9dad-603eaf90ee00 service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:05.102486 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe40dda-6d7d-464c-9479-4221167b7e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:05.103155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe40dda-6d7d-464c-9479-4221167b7e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:05.103433 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe40dda-6d7d-464c-9479-4221167b7e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:05.103705 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe40dda-6d7d-464c-9479-4221167b7e7a admin admin] PATH_INFO: `/v3/users/6880fdaab78545af91954a8df86ee06a` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:05.104949 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe40dda-6d7d-464c-9479-4221167b7e7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6880fdaab78545af91954a8df86ee06a)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:05.106861 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe40dda-6d7d-464c-9479-4221167b7e7a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:05.150006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:05.171525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-cfe40dda-6d7d-464c-9479-4221167b7e7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6880fdaab78545af91954a8df86ee06a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:05.182456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-cfe40dda-6d7d-464c-9479-4221167b7e7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6880fdaab78545af91954a8df86ee06a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:05.198944 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-cfe40dda-6d7d-464c-9479-4221167b7e7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6880fdaab78545af91954a8df86ee06a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:05.208308 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-cfe40dda-6d7d-464c-9479-4221167b7e7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6880fdaab78545af91954a8df86ee06a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:05.210245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-cfe40dda-6d7d-464c-9479-4221167b7e7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6880fdaab78545af91954a8df86ee06a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:05.216336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1223/2458] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:59:05 2022] DELETE /identity/v3/users/6880fdaab78545af91954a8df86ee06a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:05.221504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cb1d9-9d51-470e-a44c-bb5144f9eacf None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:05.221947 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cb1d9-9d51-470e-a44c-bb5144f9eacf None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:05.222109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cb1d9-9d51-470e-a44c-bb5144f9eacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:05.243306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-f56cb1d9-9d51-470e-a44c-bb5144f9eacf None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:59:05.244873 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f56cb1d9-9d51-470e-a44c-bb5144f9eacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:05.318774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1236/2459] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:59:05.352890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe40dda-6d7d-464c-9479-4221167b7e7a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:05.355446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-cfe40dda-6d7d-464c-9479-4221167b7e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:05.426606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe40dda-6d7d-464c-9479-4221167b7e7a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:05.431882 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51599f69-01c4-416b-afce-e4aef8db6c1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:05.433356 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51599f69-01c4-416b-afce-e4aef8db6c1c service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:05.433775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51599f69-01c4-416b-afce-e4aef8db6c1c service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:05.434168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51599f69-01c4-416b-afce-e4aef8db6c1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:05.448930 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51599f69-01c4-416b-afce-e4aef8db6c1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:05.450532 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51599f69-01c4-416b-afce-e4aef8db6c1c service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:05.462638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1224/2460] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:59:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:59:06.261929 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cb1d9-9d51-470e-a44c-bb5144f9eacf None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:06.263493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f56cb1d9-9d51-470e-a44c-bb5144f9eacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:06.288899 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cb1d9-9d51-470e-a44c-bb5144f9eacf None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:06.297273 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc99ca-34c7-4990-8ae1-7482fabb3547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:06.297952 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc99ca-34c7-4990-8ae1-7482fabb3547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:06.298234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc99ca-34c7-4990-8ae1-7482fabb3547 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:06.299118 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc99ca-34c7-4990-8ae1-7482fabb3547 admin admin] PATH_INFO: `/v3/projects/d0d1ec9f5bb14df6a0335c2993c065f8` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:06.304030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc99ca-34c7-4990-8ae1-7482fabb3547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0d1ec9f5bb14df6a0335c2993c065f8)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:06.306031 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc99ca-34c7-4990-8ae1-7482fabb3547 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:06.355809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c5dc99ca-34c7-4990-8ae1-7482fabb3547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0d1ec9f5bb14df6a0335c2993c065f8', 'request_id': 'req-c5dc99ca-34c7-4990-8ae1-7482fabb3547'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:06.362891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c5dc99ca-34c7-4990-8ae1-7482fabb3547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0d1ec9f5bb14df6a0335c2993c065f8', 'request_id': 'req-c5dc99ca-34c7-4990-8ae1-7482fabb3547'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:06.386961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c5dc99ca-34c7-4990-8ae1-7482fabb3547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0d1ec9f5bb14df6a0335c2993c065f8', 'request_id': 'req-c5dc99ca-34c7-4990-8ae1-7482fabb3547'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:06.399651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:06.408870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c5dc99ca-34c7-4990-8ae1-7482fabb3547 admin admin] The token cache is being invalidate because project d0d1ec9f5bb14df6a0335c2993c065f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:59:06.409258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c5dc99ca-34c7-4990-8ae1-7482fabb3547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:06.414699 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c5dc99ca-34c7-4990-8ae1-7482fabb3547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:06.424976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1237/2461] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:59:06 2022] DELETE /identity/v3/projects/d0d1ec9f5bb14df6a0335c2993c065f8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:06.432991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51599f69-01c4-416b-afce-e4aef8db6c1c service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:06.447549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51599f69-01c4-416b-afce-e4aef8db6c1c service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:06.530422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28864774-b2ce-49df-b98a-f86b92984852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:06.531392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28864774-b2ce-49df-b98a-f86b92984852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:06.531891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28864774-b2ce-49df-b98a-f86b92984852 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:06.532205 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28864774-b2ce-49df-b98a-f86b92984852 admin admin] PATH_INFO: `/v3/users/92aba7679ade4116a9095ac54e000ae4` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:06.534847 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28864774-b2ce-49df-b98a-f86b92984852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92aba7679ade4116a9095ac54e000ae4)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:06.537677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28864774-b2ce-49df-b98a-f86b92984852 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:06.606838 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-28864774-b2ce-49df-b98a-f86b92984852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92aba7679ade4116a9095ac54e000ae4'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:06.617335 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-28864774-b2ce-49df-b98a-f86b92984852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92aba7679ade4116a9095ac54e000ae4'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:06.620730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:06.643103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-28864774-b2ce-49df-b98a-f86b92984852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92aba7679ade4116a9095ac54e000ae4'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:06.652656 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-28864774-b2ce-49df-b98a-f86b92984852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92aba7679ade4116a9095ac54e000ae4'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:06.653481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-28864774-b2ce-49df-b98a-f86b92984852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92aba7679ade4116a9095ac54e000ae4'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:06.660317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1225/2462] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:59:06 2022] DELETE /identity/v3/users/92aba7679ade4116a9095ac54e000ae4 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:07.029113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc99ca-34c7-4990-8ae1-7482fabb3547 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:07.030972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c5dc99ca-34c7-4990-8ae1-7482fabb3547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:07.057302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc99ca-34c7-4990-8ae1-7482fabb3547 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:07.133165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cbf6c1-6d89-4d36-9906-e7f12aa62be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:07.134386 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbf6c1-6d89-4d36-9906-e7f12aa62be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:07.134639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbf6c1-6d89-4d36-9906-e7f12aa62be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:07.134945 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbf6c1-6d89-4d36-9906-e7f12aa62be7 admin admin] PATH_INFO: `/v3/projects/5dbea53f2b4c484cb96fcf350cd366b8` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:07.136154 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cbf6c1-6d89-4d36-9906-e7f12aa62be7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dbea53f2b4c484cb96fcf350cd366b8)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:07.138442 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cbf6c1-6d89-4d36-9906-e7f12aa62be7 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:07.203138 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e4cbf6c1-6d89-4d36-9906-e7f12aa62be7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dbea53f2b4c484cb96fcf350cd366b8', 'request_id': 'req-e4cbf6c1-6d89-4d36-9906-e7f12aa62be7'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:07.211371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e4cbf6c1-6d89-4d36-9906-e7f12aa62be7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dbea53f2b4c484cb96fcf350cd366b8', 'request_id': 'req-e4cbf6c1-6d89-4d36-9906-e7f12aa62be7'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:07.222802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e4cbf6c1-6d89-4d36-9906-e7f12aa62be7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dbea53f2b4c484cb96fcf350cd366b8', 'request_id': 'req-e4cbf6c1-6d89-4d36-9906-e7f12aa62be7'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:07.243000 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e4cbf6c1-6d89-4d36-9906-e7f12aa62be7 admin admin] The token cache is being invalidate because project 5dbea53f2b4c484cb96fcf350cd366b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:59:07.243330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e4cbf6c1-6d89-4d36-9906-e7f12aa62be7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:07.244550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e4cbf6c1-6d89-4d36-9906-e7f12aa62be7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:07.246290 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1238/2463] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:59:07 2022] DELETE /identity/v3/projects/5dbea53f2b4c484cb96fcf350cd366b8 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:59:07.252944 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c972cb-eeb1-4242-ad6e-73df89a53a9f None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:07.253179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c972cb-eeb1-4242-ad6e-73df89a53a9f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:07.253405 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c972cb-eeb1-4242-ad6e-73df89a53a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:07.284057 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-a0c972cb-eeb1-4242-ad6e-73df89a53a9f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:59:07.285584 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a0c972cb-eeb1-4242-ad6e-73df89a53a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:07.365837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1226/2464] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:59:07.370134 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f4640-4063-4c96-b15e-60d5e5596c0a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:07.370596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f4640-4063-4c96-b15e-60d5e5596c0a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:07.370816 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f4640-4063-4c96-b15e-60d5e5596c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:07.392200 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-1e3f4640-4063-4c96-b15e-60d5e5596c0a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:59:07.393518 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1e3f4640-4063-4c96-b15e-60d5e5596c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:07.403304 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1239/2465] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:59:07.408882 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c972cb-eeb1-4242-ad6e-73df89a53a9f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:07.418707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c972cb-eeb1-4242-ad6e-73df89a53a9f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:07.433615 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a504b-0c05-40a4-9a8d-c5c5cd684a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:07.434516 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a504b-0c05-40a4-9a8d-c5c5cd684a37 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:07.434896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a504b-0c05-40a4-9a8d-c5c5cd684a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:07.435239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a504b-0c05-40a4-9a8d-c5c5cd684a37 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:07.436548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a504b-0c05-40a4-9a8d-c5c5cd684a37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:59:07.436548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a504b-0c05-40a4-9a8d-c5c5cd684a37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:07.439117 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a504b-0c05-40a4-9a8d-c5c5cd684a37 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:07.453095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1227/2466] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 18:59:07.457683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f4640-4063-4c96-b15e-60d5e5596c0a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:07.467281 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f4640-4063-4c96-b15e-60d5e5596c0a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:07.473476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301a7e4-e577-4c08-a557-355b319273af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:07.474034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301a7e4-e577-4c08-a557-355b319273af admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:07.474323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301a7e4-e577-4c08-a557-355b319273af admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:07.474598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301a7e4-e577-4c08-a557-355b319273af admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:07.475302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301a7e4-e577-4c08-a557-355b319273af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:07.477301 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301a7e4-e577-4c08-a557-355b319273af admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:07.510840 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1240/2467] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:59:07 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:59:07.514563 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301a7e4-e577-4c08-a557-355b319273af admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:07.514703 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a504b-0c05-40a4-9a8d-c5c5cd684a37 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:07.516770 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c301a7e4-e577-4c08-a557-355b319273af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:07.527544 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a504b-0c05-40a4-9a8d-c5c5cd684a37 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:07.592808 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301a7e4-e577-4c08-a557-355b319273af admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:07.598855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f6dc5-9f06-45b9-a22a-1dd0dc5a46a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:07.599518 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f6dc5-9f06-45b9-a22a-1dd0dc5a46a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:07.599799 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f6dc5-9f06-45b9-a22a-1dd0dc5a46a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:07.600076 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f6dc5-9f06-45b9-a22a-1dd0dc5a46a6 admin admin] PATH_INFO: `/v3/users/effc38ce5cd6441b92ed3bbb2eaee76c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:07.601301 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f6dc5-9f06-45b9-a22a-1dd0dc5a46a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=effc38ce5cd6441b92ed3bbb2eaee76c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:07.603255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f6dc5-9f06-45b9-a22a-1dd0dc5a46a6 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:07.605275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04177f-2db2-45a4-92b0-b50e6da44818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:07.605421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:07.605535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04177f-2db2-45a4-92b0-b50e6da44818 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:07.605535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04177f-2db2-45a4-92b0-b50e6da44818 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:07.605997 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04177f-2db2-45a4-92b0-b50e6da44818 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:07.606781 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04177f-2db2-45a4-92b0-b50e6da44818 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:07.608736 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04177f-2db2-45a4-92b0-b50e6da44818 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:07.645449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1228/2468] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:59:07 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:59:07.649798 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04177f-2db2-45a4-92b0-b50e6da44818 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:07.661422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04177f-2db2-45a4-92b0-b50e6da44818 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:07.666317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e116b39f-fc3d-48fe-a782-e2361e3db03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:07.667014 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116b39f-fc3d-48fe-a782-e2361e3db03b admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:07.667278 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-df5f6dc5-9f06-45b9-a22a-1dd0dc5a46a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'effc38ce5cd6441b92ed3bbb2eaee76c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:07.667540 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116b39f-fc3d-48fe-a782-e2361e3db03b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:07.667892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116b39f-fc3d-48fe-a782-e2361e3db03b admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:07.668745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e116b39f-fc3d-48fe-a782-e2361e3db03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:07.670426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e116b39f-fc3d-48fe-a782-e2361e3db03b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:07.678685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-df5f6dc5-9f06-45b9-a22a-1dd0dc5a46a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'effc38ce5cd6441b92ed3bbb2eaee76c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:07.683818 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1229/2469] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:59:07.687424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e116b39f-fc3d-48fe-a782-e2361e3db03b admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:07.698137 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e116b39f-fc3d-48fe-a782-e2361e3db03b admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:07.699052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-df5f6dc5-9f06-45b9-a22a-1dd0dc5a46a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'effc38ce5cd6441b92ed3bbb2eaee76c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:07.703060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685eddc-0039-487b-8fc6-7831f2d84c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:07.703747 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685eddc-0039-487b-8fc6-7831f2d84c25 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:07.704110 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685eddc-0039-487b-8fc6-7831f2d84c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:07.704664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685eddc-0039-487b-8fc6-7831f2d84c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:07.705542 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685eddc-0039-487b-8fc6-7831f2d84c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:07.707311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685eddc-0039-487b-8fc6-7831f2d84c25 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:07.710403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-df5f6dc5-9f06-45b9-a22a-1dd0dc5a46a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'effc38ce5cd6441b92ed3bbb2eaee76c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:07.711659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-df5f6dc5-9f06-45b9-a22a-1dd0dc5a46a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'effc38ce5cd6441b92ed3bbb2eaee76c'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:07.721738 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1230/2470] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:59:07.722303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1241/2471] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:59:07 2022] DELETE /identity/v3/users/effc38ce5cd6441b92ed3bbb2eaee76c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:07.723245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685eddc-0039-487b-8fc6-7831f2d84c25 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:07.725012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-3685eddc-0039-487b-8fc6-7831f2d84c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:07.740325 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685eddc-0039-487b-8fc6-7831f2d84c25 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:07.769927 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:07.814401 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bfede-a47d-4591-aea2-01c56421eb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:07.816004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bfede-a47d-4591-aea2-01c56421eb28 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:07.816507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bfede-a47d-4591-aea2-01c56421eb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:07.818159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bfede-a47d-4591-aea2-01c56421eb28 admin admin] PATH_INFO: `/v3/projects/9c29afde7b4647d49c924eec1ecd57b3/users/20ca84850de14578ba05a38b3d326450/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:07.836794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bfede-a47d-4591-aea2-01c56421eb28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c29afde7b4647d49c924eec1ecd57b3, user_id=20ca84850de14578ba05a38b3d326450, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:07.839590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bfede-a47d-4591-aea2-01c56421eb28 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:07.855424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1231/2472] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:59:07 2022] PUT /identity/v3/projects/9c29afde7b4647d49c924eec1ecd57b3/users/20ca84850de14578ba05a38b3d326450/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:07.866322 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55975e75-a3e9-4a7d-bb08-a56ddc5af52f None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:07.866610 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55975e75-a3e9-4a7d-bb08-a56ddc5af52f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:07.866876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55975e75-a3e9-4a7d-bb08-a56ddc5af52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:07.899587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-55975e75-a3e9-4a7d-bb08-a56ddc5af52f None None] MFA Rules not processed for user `20ca84850de14578ba05a38b3d326450`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:59:07.900383 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-55975e75-a3e9-4a7d-bb08-a56ddc5af52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:07.937397 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bfede-a47d-4591-aea2-01c56421eb28 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:07.940937 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-182bfede-a47d-4591-aea2-01c56421eb28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:07.997339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1242/2473] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:59:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:59:08.009266 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55975e75-a3e9-4a7d-bb08-a56ddc5af52f None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:08.011731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-55975e75-a3e9-4a7d-bb08-a56ddc5af52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:08.028426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bfede-a47d-4591-aea2-01c56421eb28 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:08.034373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fe738-d222-4141-9468-2f707db22dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:08.035112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fe738-d222-4141-9468-2f707db22dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:08.035404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fe738-d222-4141-9468-2f707db22dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:08.035680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fe738-d222-4141-9468-2f707db22dd7 admin admin] PATH_INFO: `/v3/projects/182c4b8842bb4aa1a31fbd603c52b276` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:08.036956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fe738-d222-4141-9468-2f707db22dd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=182c4b8842bb4aa1a31fbd603c52b276)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:08.038900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fe738-d222-4141-9468-2f707db22dd7 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:08.093388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55975e75-a3e9-4a7d-bb08-a56ddc5af52f None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:08.100029 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c30fe738-d222-4141-9468-2f707db22dd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '182c4b8842bb4aa1a31fbd603c52b276', 'request_id': 'req-c30fe738-d222-4141-9468-2f707db22dd7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:08.106215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c30fe738-d222-4141-9468-2f707db22dd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '182c4b8842bb4aa1a31fbd603c52b276', 'request_id': 'req-c30fe738-d222-4141-9468-2f707db22dd7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:08.113256 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c30fe738-d222-4141-9468-2f707db22dd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '182c4b8842bb4aa1a31fbd603c52b276', 'request_id': 'req-c30fe738-d222-4141-9468-2f707db22dd7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:08.118337 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:08.129652 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c30fe738-d222-4141-9468-2f707db22dd7 admin admin] The token cache is being invalidate because project 182c4b8842bb4aa1a31fbd603c52b276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:59:08.130046 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c30fe738-d222-4141-9468-2f707db22dd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:08.131042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c30fe738-d222-4141-9468-2f707db22dd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:08.135645 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1232/2474] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:59:07 2022] DELETE /identity/v3/projects/182c4b8842bb4aa1a31fbd603c52b276 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:08.139419 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fe738-d222-4141-9468-2f707db22dd7 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:08.141579 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c30fe738-d222-4141-9468-2f707db22dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:08.165297 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e2486-46ee-4284-bee1-fd75039b5d9f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4799b0c9e5aa47d389121d2183666d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:08.165691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e2486-46ee-4284-bee1-fd75039b5d9f service glance] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:08.165980 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e2486-46ee-4284-bee1-fd75039b5d9f service glance] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:08.166258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e2486-46ee-4284-bee1-fd75039b5d9f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:08.168404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5e7e2486-46ee-4284-bee1-fd75039b5d9f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:08.196180 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:08.224209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fe738-d222-4141-9468-2f707db22dd7 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:08.231736 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93354727-a45f-453e-a0b6-b4cb976b4b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:08.233494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93354727-a45f-453e-a0b6-b4cb976b4b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:08.233916 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93354727-a45f-453e-a0b6-b4cb976b4b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:08.234283 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93354727-a45f-453e-a0b6-b4cb976b4b28 admin admin] PATH_INFO: `/v3/users/e446a9defd8e4b19aaeb590972b735e7` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:08.235647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93354727-a45f-453e-a0b6-b4cb976b4b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e446a9defd8e4b19aaeb590972b735e7)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:08.237823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93354727-a45f-453e-a0b6-b4cb976b4b28 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:08.241862 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e2486-46ee-4284-bee1-fd75039b5d9f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:08.243359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e2486-46ee-4284-bee1-fd75039b5d9f service glance] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:08.255240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1243/2475] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:59:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:59:08.302030 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-93354727-a45f-453e-a0b6-b4cb976b4b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e446a9defd8e4b19aaeb590972b735e7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:08.310552 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-93354727-a45f-453e-a0b6-b4cb976b4b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e446a9defd8e4b19aaeb590972b735e7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:08.329394 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-93354727-a45f-453e-a0b6-b4cb976b4b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e446a9defd8e4b19aaeb590972b735e7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:08.347647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-93354727-a45f-453e-a0b6-b4cb976b4b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e446a9defd8e4b19aaeb590972b735e7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:08.348460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-93354727-a45f-453e-a0b6-b4cb976b4b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e446a9defd8e4b19aaeb590972b735e7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:08.356278 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1233/2476] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:59:08 2022] DELETE /identity/v3/users/e446a9defd8e4b19aaeb590972b735e7 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:08.727789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e2486-46ee-4284-bee1-fd75039b5d9f service glance] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:08.747166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e2486-46ee-4284-bee1-fd75039b5d9f service glance] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:08.862641 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c25ed1-a390-4acc-a38c-7e4c8eaf6fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:08.863859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c25ed1-a390-4acc-a38c-7e4c8eaf6fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:08.864503 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c25ed1-a390-4acc-a38c-7e4c8eaf6fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:08.865021 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c25ed1-a390-4acc-a38c-7e4c8eaf6fa4 admin admin] PATH_INFO: `/v3/projects/0b9346f2f32d4733bec958c53797e88b` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:08.866598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c25ed1-a390-4acc-a38c-7e4c8eaf6fa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b9346f2f32d4733bec958c53797e88b)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:08.869286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c25ed1-a390-4acc-a38c-7e4c8eaf6fa4 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:08.933151 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-87c25ed1-a390-4acc-a38c-7e4c8eaf6fa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b9346f2f32d4733bec958c53797e88b', 'request_id': 'req-87c25ed1-a390-4acc-a38c-7e4c8eaf6fa4'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:08.947235 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-87c25ed1-a390-4acc-a38c-7e4c8eaf6fa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b9346f2f32d4733bec958c53797e88b', 'request_id': 'req-87c25ed1-a390-4acc-a38c-7e4c8eaf6fa4'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:08.957795 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-87c25ed1-a390-4acc-a38c-7e4c8eaf6fa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b9346f2f32d4733bec958c53797e88b', 'request_id': 'req-87c25ed1-a390-4acc-a38c-7e4c8eaf6fa4'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:08.978157 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-87c25ed1-a390-4acc-a38c-7e4c8eaf6fa4 admin admin] The token cache is being invalidate because project 0b9346f2f32d4733bec958c53797e88b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:59:08.978698 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-87c25ed1-a390-4acc-a38c-7e4c8eaf6fa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:08.979363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-87c25ed1-a390-4acc-a38c-7e4c8eaf6fa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:08.986586 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1244/2477] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:59:08 2022] DELETE /identity/v3/projects/0b9346f2f32d4733bec958c53797e88b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:18.354531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=34,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:18.974228 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:34.483439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93354727-a45f-453e-a0b6-b4cb976b4b28 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:34.484915 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-93354727-a45f-453e-a0b6-b4cb976b4b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:34.498086 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93354727-a45f-453e-a0b6-b4cb976b4b28 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:34.556828 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50160df-63f3-4b73-9db1-6597b8d6cfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:34.557473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50160df-63f3-4b73-9db1-6597b8d6cfa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:34.557755 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50160df-63f3-4b73-9db1-6597b8d6cfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:34.558030 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50160df-63f3-4b73-9db1-6597b8d6cfa6 admin admin] PATH_INFO: `/v3/users/20ca84850de14578ba05a38b3d326450` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:34.559225 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50160df-63f3-4b73-9db1-6597b8d6cfa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20ca84850de14578ba05a38b3d326450)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:34.561184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50160df-63f3-4b73-9db1-6597b8d6cfa6 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:34.607524 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e50160df-63f3-4b73-9db1-6597b8d6cfa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20ca84850de14578ba05a38b3d326450'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:34.615005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e50160df-63f3-4b73-9db1-6597b8d6cfa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20ca84850de14578ba05a38b3d326450'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:34.627475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e50160df-63f3-4b73-9db1-6597b8d6cfa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20ca84850de14578ba05a38b3d326450'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:34.636456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e50160df-63f3-4b73-9db1-6597b8d6cfa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20ca84850de14578ba05a38b3d326450'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:34.637141 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-e50160df-63f3-4b73-9db1-6597b8d6cfa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20ca84850de14578ba05a38b3d326450'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:34.643020 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1234/2478] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:59:34 2022] DELETE /identity/v3/users/20ca84850de14578ba05a38b3d326450 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:34.648432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc0024-ae8d-469f-bf82-2e75376729bc None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:34.649001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc0024-ae8d-469f-bf82-2e75376729bc None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:34.649001 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc0024-ae8d-469f-bf82-2e75376729bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:34.668415 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-fdbc0024-ae8d-469f-bf82-2e75376729bc None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:59:34.669646 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-fdbc0024-ae8d-469f-bf82-2e75376729bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:34.730115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1245/2479] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:59:34.739705 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50160df-63f3-4b73-9db1-6597b8d6cfa6 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:34.741347 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e50160df-63f3-4b73-9db1-6597b8d6cfa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:34.799344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50160df-63f3-4b73-9db1-6597b8d6cfa6 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:34.803013 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3503d9-28e4-4d34-afc2-d314c7f2c95a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:34.803633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3503d9-28e4-4d34-afc2-d314c7f2c95a service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:34.803914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3503d9-28e4-4d34-afc2-d314c7f2c95a service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:34.804190 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3503d9-28e4-4d34-afc2-d314c7f2c95a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:34.813326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3503d9-28e4-4d34-afc2-d314c7f2c95a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:34.814741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3503d9-28e4-4d34-afc2-d314c7f2c95a service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:34.824226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1235/2480] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:59:35.092084 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc0024-ae8d-469f-bf82-2e75376729bc None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:35.093642 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-fdbc0024-ae8d-469f-bf82-2e75376729bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:35.106140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc0024-ae8d-469f-bf82-2e75376729bc None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:35.111043 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d848423-a07f-4bda-af52-9a17cb44c9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:35.111699 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d848423-a07f-4bda-af52-9a17cb44c9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:35.112015 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d848423-a07f-4bda-af52-9a17cb44c9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:35.112364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d848423-a07f-4bda-af52-9a17cb44c9ff admin admin] PATH_INFO: `/v3/projects/9c29afde7b4647d49c924eec1ecd57b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:35.113552 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d848423-a07f-4bda-af52-9a17cb44c9ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c29afde7b4647d49c924eec1ecd57b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:35.115964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d848423-a07f-4bda-af52-9a17cb44c9ff admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:35.165934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5d848423-a07f-4bda-af52-9a17cb44c9ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c29afde7b4647d49c924eec1ecd57b3', 'request_id': 'req-5d848423-a07f-4bda-af52-9a17cb44c9ff'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:35.171799 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5d848423-a07f-4bda-af52-9a17cb44c9ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c29afde7b4647d49c924eec1ecd57b3', 'request_id': 'req-5d848423-a07f-4bda-af52-9a17cb44c9ff'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:35.179554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5d848423-a07f-4bda-af52-9a17cb44c9ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c29afde7b4647d49c924eec1ecd57b3', 'request_id': 'req-5d848423-a07f-4bda-af52-9a17cb44c9ff'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:35.190677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5d848423-a07f-4bda-af52-9a17cb44c9ff admin admin] The token cache is being invalidate because project 9c29afde7b4647d49c924eec1ecd57b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:59:35.190963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5d848423-a07f-4bda-af52-9a17cb44c9ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:35.191586 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-5d848423-a07f-4bda-af52-9a17cb44c9ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:35.193117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1246/2481] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:59:35 2022] DELETE /identity/v3/projects/9c29afde7b4647d49c924eec1ecd57b3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:35.198931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400317f1-3826-4864-bce4-844c7bc6884a None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:35.199272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400317f1-3826-4864-bce4-844c7bc6884a None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:35.199556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400317f1-3826-4864-bce4-844c7bc6884a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:35.219938 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-400317f1-3826-4864-bce4-844c7bc6884a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:59:35.221213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-400317f1-3826-4864-bce4-844c7bc6884a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:35.238783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:35.283816 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1236/2482] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:59:35.291460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ab6a2-c14a-43fb-94ac-6e4f9598daf3 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:35.291757 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ab6a2-c14a-43fb-94ac-6e4f9598daf3 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:35.292037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ab6a2-c14a-43fb-94ac-6e4f9598daf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:35.311906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-779ab6a2-c14a-43fb-94ac-6e4f9598daf3 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:59:35.313262 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-779ab6a2-c14a-43fb-94ac-6e4f9598daf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:35.322003 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1247/2483] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:59:35.326262 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400317f1-3826-4864-bce4-844c7bc6884a None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:35.337064 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400317f1-3826-4864-bce4-844c7bc6884a None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:35.341668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5a369-f4d5-4bd1-a171-b86a811b9a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:35.342286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5a369-f4d5-4bd1-a171-b86a811b9a2b admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:35.342569 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5a369-f4d5-4bd1-a171-b86a811b9a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:35.342846 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5a369-f4d5-4bd1-a171-b86a811b9a2b admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:35.343545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5a369-f4d5-4bd1-a171-b86a811b9a2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:59:35.343850 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5a369-f4d5-4bd1-a171-b86a811b9a2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:35.345493 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5a369-f4d5-4bd1-a171-b86a811b9a2b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:35.357205 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1237/2484] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:59:35.361611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ab6a2-c14a-43fb-94ac-6e4f9598daf3 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:35.372823 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ab6a2-c14a-43fb-94ac-6e4f9598daf3 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:35.377701 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a91040-b4b9-44f5-80fd-3b3e415dd762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:35.378308 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a91040-b4b9-44f5-80fd-3b3e415dd762 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:35.378586 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a91040-b4b9-44f5-80fd-3b3e415dd762 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:35.378852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a91040-b4b9-44f5-80fd-3b3e415dd762 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:35.379508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a91040-b4b9-44f5-80fd-3b3e415dd762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:35.382630 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a91040-b4b9-44f5-80fd-3b3e415dd762 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:35.414141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1248/2485] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:59:35 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:59:35.418480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5a369-f4d5-4bd1-a171-b86a811b9a2b admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:35.429333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5a369-f4d5-4bd1-a171-b86a811b9a2b admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:35.486001 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df160d02-f78a-4f76-a5cc-5450801a6a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:35.486638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df160d02-f78a-4f76-a5cc-5450801a6a44 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:35.486885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df160d02-f78a-4f76-a5cc-5450801a6a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:35.487156 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df160d02-f78a-4f76-a5cc-5450801a6a44 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:35.487823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df160d02-f78a-4f76-a5cc-5450801a6a44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:35.489753 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df160d02-f78a-4f76-a5cc-5450801a6a44 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:35.512239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1238/2486] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:59:35 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:59:35.516367 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a91040-b4b9-44f5-80fd-3b3e415dd762 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:35.527352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a91040-b4b9-44f5-80fd-3b3e415dd762 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:35.531957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540e3c8-a8c0-4fed-899c-312c84d2152e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:35.532599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540e3c8-a8c0-4fed-899c-312c84d2152e admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:35.532882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540e3c8-a8c0-4fed-899c-312c84d2152e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:35.533160 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540e3c8-a8c0-4fed-899c-312c84d2152e admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:35.533858 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540e3c8-a8c0-4fed-899c-312c84d2152e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:35.535449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540e3c8-a8c0-4fed-899c-312c84d2152e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:35.546007 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1249/2487] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:59:35.550027 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df160d02-f78a-4f76-a5cc-5450801a6a44 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:35.560401 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df160d02-f78a-4f76-a5cc-5450801a6a44 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:35.565445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e0f16-b4b4-40a2-8ea0-209680cc0aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:35.566054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e0f16-b4b4-40a2-8ea0-209680cc0aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:35.566341 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e0f16-b4b4-40a2-8ea0-209680cc0aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:35.566616 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e0f16-b4b4-40a2-8ea0-209680cc0aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:35.567332 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e0f16-b4b4-40a2-8ea0-209680cc0aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:35.569005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e0f16-b4b4-40a2-8ea0-209680cc0aa7 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:35.579787 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1239/2488] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 18:59:35.584623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540e3c8-a8c0-4fed-899c-312c84d2152e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:35.594822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540e3c8-a8c0-4fed-899c-312c84d2152e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:35.600189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8583ec-8b51-4d25-882e-1555ce3fe1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:35.600900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8583ec-8b51-4d25-882e-1555ce3fe1e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:35.601184 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8583ec-8b51-4d25-882e-1555ce3fe1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:35.601459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8583ec-8b51-4d25-882e-1555ce3fe1e9 admin admin] PATH_INFO: `/v3/projects/d455239c133d4d0b82dd52875541b66e/users/d557543b4ae441cdac9e849c8022d828/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:35.616974 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8583ec-8b51-4d25-882e-1555ce3fe1e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d455239c133d4d0b82dd52875541b66e, user_id=d557543b4ae441cdac9e849c8022d828, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:35.619644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8583ec-8b51-4d25-882e-1555ce3fe1e9 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:35.633590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1250/2489] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:59:35 2022] PUT /identity/v3/projects/d455239c133d4d0b82dd52875541b66e/users/d557543b4ae441cdac9e849c8022d828/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 18:59:35.700266 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d013244-719e-426c-9057-ff8fccb21d00 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:35.700481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d013244-719e-426c-9057-ff8fccb21d00 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:35.700766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d013244-719e-426c-9057-ff8fccb21d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:35.727878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-8d013244-719e-426c-9057-ff8fccb21d00 None None] MFA Rules not processed for user `d557543b4ae441cdac9e849c8022d828`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:59:35.739073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-8d013244-719e-426c-9057-ff8fccb21d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:35.821167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1240/2490] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:59:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:59:35.825621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8583ec-8b51-4d25-882e-1555ce3fe1e9 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:35.846377 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:35.889008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8583ec-8b51-4d25-882e-1555ce3fe1e9 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:35.893690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b8e4e-9693-45e4-911a-69dbeed15d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:35.894301 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b8e4e-9693-45e4-911a-69dbeed15d47 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:35.894580 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b8e4e-9693-45e4-911a-69dbeed15d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:35.894850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b8e4e-9693-45e4-911a-69dbeed15d47 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:35.895518 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b8e4e-9693-45e4-911a-69dbeed15d47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:35.897541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b8e4e-9693-45e4-911a-69dbeed15d47 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:35.924874 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1251/2491] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:59:35 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:59:35.929562 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d013244-719e-426c-9057-ff8fccb21d00 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:35.940213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d013244-719e-426c-9057-ff8fccb21d00 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:35.953050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:35.999939 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35046e35-a4da-4e5f-8685-c0edf2a6edc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:36.000857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35046e35-a4da-4e5f-8685-c0edf2a6edc5 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:36.001142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35046e35-a4da-4e5f-8685-c0edf2a6edc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:36.001415 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35046e35-a4da-4e5f-8685-c0edf2a6edc5 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:36.002082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35046e35-a4da-4e5f-8685-c0edf2a6edc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:36.003994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35046e35-a4da-4e5f-8685-c0edf2a6edc5 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:36.026518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1241/2492] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:59:35 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:59:36.030143 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b8e4e-9693-45e4-911a-69dbeed15d47 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:36.046046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b8e4e-9693-45e4-911a-69dbeed15d47 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:36.050783 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df370e-5442-4a0d-9b47-8172dbf83c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:36.051404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df370e-5442-4a0d-9b47-8172dbf83c1b admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:36.051685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df370e-5442-4a0d-9b47-8172dbf83c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:36.051964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df370e-5442-4a0d-9b47-8172dbf83c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:36.053364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df370e-5442-4a0d-9b47-8172dbf83c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:36.055054 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df370e-5442-4a0d-9b47-8172dbf83c1b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:36.065912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1252/2493] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:59:36.070658 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35046e35-a4da-4e5f-8685-c0edf2a6edc5 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:36.088635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35046e35-a4da-4e5f-8685-c0edf2a6edc5 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:36.093434 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af4408-e260-433d-9453-14a8c45b8d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:36.094056 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af4408-e260-433d-9453-14a8c45b8d1c admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:36.094337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af4408-e260-433d-9453-14a8c45b8d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:36.094638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af4408-e260-433d-9453-14a8c45b8d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:36.095336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af4408-e260-433d-9453-14a8c45b8d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:36.096963 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af4408-e260-433d-9453-14a8c45b8d1c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:36.108387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1242/2494] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:59:36.112809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df370e-5442-4a0d-9b47-8172dbf83c1b admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:36.123727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df370e-5442-4a0d-9b47-8172dbf83c1b admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:36.128846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb8902-f025-4122-b8ad-f47bbcc5d84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:36.129509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb8902-f025-4122-b8ad-f47bbcc5d84d admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:36.129791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb8902-f025-4122-b8ad-f47bbcc5d84d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:36.130067 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb8902-f025-4122-b8ad-f47bbcc5d84d admin admin] PATH_INFO: `/v3/projects/a408c9ba4a1349aeaf5c447dcfb9a9b4/users/0198409e2d924bf49eef94ea878da47e/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:36.146891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb8902-f025-4122-b8ad-f47bbcc5d84d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a408c9ba4a1349aeaf5c447dcfb9a9b4, user_id=0198409e2d924bf49eef94ea878da47e, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:36.149592 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb8902-f025-4122-b8ad-f47bbcc5d84d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:36.164020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1253/2495] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:59:36 2022] PUT /identity/v3/projects/a408c9ba4a1349aeaf5c447dcfb9a9b4/users/0198409e2d924bf49eef94ea878da47e/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:36.174336 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e855eb-11c9-4bb3-beaf-31e7b8ac13d4 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:36.174636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e855eb-11c9-4bb3-beaf-31e7b8ac13d4 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:36.174899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e855eb-11c9-4bb3-beaf-31e7b8ac13d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:36.202900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-47e855eb-11c9-4bb3-beaf-31e7b8ac13d4 None None] MFA Rules not processed for user `0198409e2d924bf49eef94ea878da47e`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:59:36.204147 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-47e855eb-11c9-4bb3-beaf-31e7b8ac13d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:36.266923 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1243/2496] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:59:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:59:36.271583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb8902-f025-4122-b8ad-f47bbcc5d84d admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:36.334117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb8902-f025-4122-b8ad-f47bbcc5d84d admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:36.338912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab8115-932a-4303-861f-2c0afc7a57c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:36.339528 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab8115-932a-4303-861f-2c0afc7a57c1 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:36.339812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab8115-932a-4303-861f-2c0afc7a57c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:36.340088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab8115-932a-4303-861f-2c0afc7a57c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:36.340786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab8115-932a-4303-861f-2c0afc7a57c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:36.343106 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab8115-932a-4303-861f-2c0afc7a57c1 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:36.372259 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1254/2497] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:59:36 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:59:36.377042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e855eb-11c9-4bb3-beaf-31e7b8ac13d4 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:36.388460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e855eb-11c9-4bb3-beaf-31e7b8ac13d4 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:36.446088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5783b4-ecb2-4056-9268-74449a7a6173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:36.446735 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5783b4-ecb2-4056-9268-74449a7a6173 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:36.447020 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5783b4-ecb2-4056-9268-74449a7a6173 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:36.447295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5783b4-ecb2-4056-9268-74449a7a6173 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:36.447980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5783b4-ecb2-4056-9268-74449a7a6173 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:36.459825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5783b4-ecb2-4056-9268-74449a7a6173 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:36.482096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1244/2498] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:59:36 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:59:36.486248 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab8115-932a-4303-861f-2c0afc7a57c1 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:36.501899 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab8115-932a-4303-861f-2c0afc7a57c1 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:36.507250 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b5d06-fa11-46c9-9e2f-bca3418a51f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:36.507861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b5d06-fa11-46c9-9e2f-bca3418a51f6 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:36.508141 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b5d06-fa11-46c9-9e2f-bca3418a51f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:36.508451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b5d06-fa11-46c9-9e2f-bca3418a51f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:36.509147 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b5d06-fa11-46c9-9e2f-bca3418a51f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:36.510745 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b5d06-fa11-46c9-9e2f-bca3418a51f6 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:36.521326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1255/2499] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:59:36.526204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5783b4-ecb2-4056-9268-74449a7a6173 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:36.536404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5783b4-ecb2-4056-9268-74449a7a6173 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:36.542174 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b055e7-8f3f-4222-894b-7bc2f4320cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:36.542833 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b055e7-8f3f-4222-894b-7bc2f4320cf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:36.543115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b055e7-8f3f-4222-894b-7bc2f4320cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:36.543395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b055e7-8f3f-4222-894b-7bc2f4320cf3 admin admin] PATH_INFO: `/v3/projects/056517beb1d444f5a0406b76794bad98/users/a768dffa369941038efc57445829d5ee/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:36.556068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:36.559311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b055e7-8f3f-4222-894b-7bc2f4320cf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=056517beb1d444f5a0406b76794bad98, user_id=a768dffa369941038efc57445829d5ee, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:36.562807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b055e7-8f3f-4222-894b-7bc2f4320cf3 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:36.576824 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1245/2500] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:59:36 2022] PUT /identity/v3/projects/056517beb1d444f5a0406b76794bad98/users/a768dffa369941038efc57445829d5ee/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:36.581122 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b5d06-fa11-46c9-9e2f-bca3418a51f6 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:36.591232 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b5d06-fa11-46c9-9e2f-bca3418a51f6 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:36.634938 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:36.651696 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8373050-4fed-421d-9f86-2b06d3c282c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:36.652372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8373050-4fed-421d-9f86-2b06d3c282c4 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:36.652679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8373050-4fed-421d-9f86-2b06d3c282c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:36.652955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8373050-4fed-421d-9f86-2b06d3c282c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:36.653650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8373050-4fed-421d-9f86-2b06d3c282c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:36.655254 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8373050-4fed-421d-9f86-2b06d3c282c4 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:36.665374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1256/2501] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:59:36.669718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b055e7-8f3f-4222-894b-7bc2f4320cf3 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:36.680197 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b055e7-8f3f-4222-894b-7bc2f4320cf3 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:36.684825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49c6bf-cacd-4d57-a2c5-af0d20761976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:36.685488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49c6bf-cacd-4d57-a2c5-af0d20761976 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:36.685774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49c6bf-cacd-4d57-a2c5-af0d20761976 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:36.686051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49c6bf-cacd-4d57-a2c5-af0d20761976 admin admin] PATH_INFO: `/v3/projects/056517beb1d444f5a0406b76794bad98/users/a768dffa369941038efc57445829d5ee/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:36.688019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49c6bf-cacd-4d57-a2c5-af0d20761976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=056517beb1d444f5a0406b76794bad98, user_id=a768dffa369941038efc57445829d5ee, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:36.690719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49c6bf-cacd-4d57-a2c5-af0d20761976 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:36.701876 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1246/2502] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:59:36 2022] PUT /identity/v3/projects/056517beb1d444f5a0406b76794bad98/users/a768dffa369941038efc57445829d5ee/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:36.713843 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cb586-a3f8-422e-9f16-c8417f889079 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:36.714056 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cb586-a3f8-422e-9f16-c8417f889079 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:36.714326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cb586-a3f8-422e-9f16-c8417f889079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:36.741323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-c55cb586-a3f8-422e-9f16-c8417f889079 None None] MFA Rules not processed for user `a768dffa369941038efc57445829d5ee`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:59:36.742530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c55cb586-a3f8-422e-9f16-c8417f889079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:36.813461 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1257/2503] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:59:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:59:36.825508 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49c6bf-cacd-4d57-a2c5-af0d20761976 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:36.827268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ea49c6bf-cacd-4d57-a2c5-af0d20761976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:36.886167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49c6bf-cacd-4d57-a2c5-af0d20761976 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:36.890028 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9917b3-e2da-43b7-8888-ba10449b3cbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:36.890654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9917b3-e2da-43b7-8888-ba10449b3cbe service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:36.890937 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9917b3-e2da-43b7-8888-ba10449b3cbe service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:36.891213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9917b3-e2da-43b7-8888-ba10449b3cbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:36.950275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9917b3-e2da-43b7-8888-ba10449b3cbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:36.951764 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9917b3-e2da-43b7-8888-ba10449b3cbe service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:36.961103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1247/2504] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:59:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:59:38.461124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cb586-a3f8-422e-9f16-c8417f889079 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:38.462518 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c55cb586-a3f8-422e-9f16-c8417f889079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:38.475589 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cb586-a3f8-422e-9f16-c8417f889079 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:38.479468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ceea25-e9b6-4978-9334-22512b0051c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:38.480072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ceea25-e9b6-4978-9334-22512b0051c9 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:38.480382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ceea25-e9b6-4978-9334-22512b0051c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:38.480653 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ceea25-e9b6-4978-9334-22512b0051c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:38.540417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ceea25-e9b6-4978-9334-22512b0051c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:38.541870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ceea25-e9b6-4978-9334-22512b0051c9 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:38.550899 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1258/2505] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:59:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:59:39.078761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9917b3-e2da-43b7-8888-ba10449b3cbe service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:39.088381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9917b3-e2da-43b7-8888-ba10449b3cbe service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:39.092076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22525349-8af2-48a0-8514-aa22769210a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:39.092735 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22525349-8af2-48a0-8514-aa22769210a8 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:39.093021 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22525349-8af2-48a0-8514-aa22769210a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:39.093323 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22525349-8af2-48a0-8514-aa22769210a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:39.102866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22525349-8af2-48a0-8514-aa22769210a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:39.104279 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22525349-8af2-48a0-8514-aa22769210a8 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:39.113780 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1248/2506] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:59:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:59:42.987379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ceea25-e9b6-4978-9334-22512b0051c9 service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:43.050745 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ceea25-e9b6-4978-9334-22512b0051c9 service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:43.056401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2342db-a37a-4ee5-9839-cdb40d8d7cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:43.057065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2342db-a37a-4ee5-9839-cdb40d8d7cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:43.057351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2342db-a37a-4ee5-9839-cdb40d8d7cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:43.057623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2342db-a37a-4ee5-9839-cdb40d8d7cc3 admin admin] PATH_INFO: `/v3/users/d557543b4ae441cdac9e849c8022d828` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:43.058777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2342db-a37a-4ee5-9839-cdb40d8d7cc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d557543b4ae441cdac9e849c8022d828)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:43.060723 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2342db-a37a-4ee5-9839-cdb40d8d7cc3 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:43.065322 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:43.110773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ea2342db-a37a-4ee5-9839-cdb40d8d7cc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd557543b4ae441cdac9e849c8022d828'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:43.118353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ea2342db-a37a-4ee5-9839-cdb40d8d7cc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd557543b4ae441cdac9e849c8022d828'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:43.130810 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ea2342db-a37a-4ee5-9839-cdb40d8d7cc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd557543b4ae441cdac9e849c8022d828'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:43.137566 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ea2342db-a37a-4ee5-9839-cdb40d8d7cc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd557543b4ae441cdac9e849c8022d828'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:43.138234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ea2342db-a37a-4ee5-9839-cdb40d8d7cc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd557543b4ae441cdac9e849c8022d828'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:43.144434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1259/2507] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:59:42 2022] DELETE /identity/v3/users/d557543b4ae441cdac9e849c8022d828 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:43.149644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca7286-4be2-4212-b947-fc6c5dbccab9 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:43.150178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca7286-4be2-4212-b947-fc6c5dbccab9 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:43.150278 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca7286-4be2-4212-b947-fc6c5dbccab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:43.170804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-efca7286-4be2-4212-b947-fc6c5dbccab9 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:59:43.171936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-efca7286-4be2-4212-b947-fc6c5dbccab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:43.230679 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1249/2508] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:59:43.240401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2342db-a37a-4ee5-9839-cdb40d8d7cc3 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:43.241821 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ea2342db-a37a-4ee5-9839-cdb40d8d7cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:43.306077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2342db-a37a-4ee5-9839-cdb40d8d7cc3 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:43.309817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13b430-82a7-40f3-a243-23b09b47bfe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:43.310454 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13b430-82a7-40f3-a243-23b09b47bfe8 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:43.310734 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13b430-82a7-40f3-a243-23b09b47bfe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:43.311166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13b430-82a7-40f3-a243-23b09b47bfe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:43.322464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c13b430-82a7-40f3-a243-23b09b47bfe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:43.324425 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c13b430-82a7-40f3-a243-23b09b47bfe8 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:43.335999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1260/2509] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 18:59:43.617219 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca7286-4be2-4212-b947-fc6c5dbccab9 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:43.619355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-efca7286-4be2-4212-b947-fc6c5dbccab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:43.636178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca7286-4be2-4212-b947-fc6c5dbccab9 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:43.641835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01300ef8-6b8c-4b55-b8ff-365e95eec712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:43.642390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01300ef8-6b8c-4b55-b8ff-365e95eec712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:43.642673 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01300ef8-6b8c-4b55-b8ff-365e95eec712 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:43.642952 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01300ef8-6b8c-4b55-b8ff-365e95eec712 admin admin] PATH_INFO: `/v3/projects/d455239c133d4d0b82dd52875541b66e` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:43.644188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01300ef8-6b8c-4b55-b8ff-365e95eec712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d455239c133d4d0b82dd52875541b66e)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:43.646706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01300ef8-6b8c-4b55-b8ff-365e95eec712 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:43.665992 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:43.698138 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-01300ef8-6b8c-4b55-b8ff-365e95eec712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd455239c133d4d0b82dd52875541b66e', 'request_id': 'req-01300ef8-6b8c-4b55-b8ff-365e95eec712'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:43.703513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-01300ef8-6b8c-4b55-b8ff-365e95eec712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd455239c133d4d0b82dd52875541b66e', 'request_id': 'req-01300ef8-6b8c-4b55-b8ff-365e95eec712'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:43.710136 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-01300ef8-6b8c-4b55-b8ff-365e95eec712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd455239c133d4d0b82dd52875541b66e', 'request_id': 'req-01300ef8-6b8c-4b55-b8ff-365e95eec712'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:43.722322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-01300ef8-6b8c-4b55-b8ff-365e95eec712 admin admin] The token cache is being invalidate because project d455239c133d4d0b82dd52875541b66e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:59:43.722615 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-01300ef8-6b8c-4b55-b8ff-365e95eec712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:43.723228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-01300ef8-6b8c-4b55-b8ff-365e95eec712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:43.725612 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1250/2510] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:59:43 2022] DELETE /identity/v3/projects/d455239c133d4d0b82dd52875541b66e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:43.729628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13b430-82a7-40f3-a243-23b09b47bfe8 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:43.739165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13b430-82a7-40f3-a243-23b09b47bfe8 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:43.796716 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d303c-3e49-43bd-ab0f-51c4be1612bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:43.797362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d303c-3e49-43bd-ab0f-51c4be1612bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:43.797665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d303c-3e49-43bd-ab0f-51c4be1612bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:43.797952 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d303c-3e49-43bd-ab0f-51c4be1612bb admin admin] PATH_INFO: `/v3/users/0198409e2d924bf49eef94ea878da47e` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:43.799140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d303c-3e49-43bd-ab0f-51c4be1612bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0198409e2d924bf49eef94ea878da47e)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:43.801099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d303c-3e49-43bd-ab0f-51c4be1612bb admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:43.830434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:43.850829 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-f04d303c-3e49-43bd-ab0f-51c4be1612bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0198409e2d924bf49eef94ea878da47e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:43.857896 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-f04d303c-3e49-43bd-ab0f-51c4be1612bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0198409e2d924bf49eef94ea878da47e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:43.871310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-f04d303c-3e49-43bd-ab0f-51c4be1612bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0198409e2d924bf49eef94ea878da47e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:43.879237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-f04d303c-3e49-43bd-ab0f-51c4be1612bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0198409e2d924bf49eef94ea878da47e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:43.880342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-f04d303c-3e49-43bd-ab0f-51c4be1612bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0198409e2d924bf49eef94ea878da47e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:43.887857 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1261/2511] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:59:43 2022] DELETE /identity/v3/users/0198409e2d924bf49eef94ea878da47e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:44.174457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01300ef8-6b8c-4b55-b8ff-365e95eec712 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:44.181235 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-01300ef8-6b8c-4b55-b8ff-365e95eec712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:44.196850 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01300ef8-6b8c-4b55-b8ff-365e95eec712 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:44.257025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:44.257975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:44.258261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:44.258540 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692 admin admin] PATH_INFO: `/v3/projects/a408c9ba4a1349aeaf5c447dcfb9a9b4` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:44.259717 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a408c9ba4a1349aeaf5c447dcfb9a9b4)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:44.261654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:44.309169 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a408c9ba4a1349aeaf5c447dcfb9a9b4', 'request_id': 'req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:44.314687 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a408c9ba4a1349aeaf5c447dcfb9a9b4', 'request_id': 'req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:44.321645 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a408c9ba4a1349aeaf5c447dcfb9a9b4', 'request_id': 'req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:44.335837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692 admin admin] The token cache is being invalidate because project a408c9ba4a1349aeaf5c447dcfb9a9b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:59:44.336126 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:44.336809 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:44.338366 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1251/2512] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:59:44 2022] DELETE /identity/v3/projects/a408c9ba4a1349aeaf5c447dcfb9a9b4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:44.342367 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d303c-3e49-43bd-ab0f-51c4be1612bb admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:44.343792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f04d303c-3e49-43bd-ab0f-51c4be1612bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:44.356719 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d303c-3e49-43bd-ab0f-51c4be1612bb admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:44.413911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270c73b-014d-494c-b7b9-16f0ad32cae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:44.414564 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270c73b-014d-494c-b7b9-16f0ad32cae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:44.414849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270c73b-014d-494c-b7b9-16f0ad32cae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:44.415126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270c73b-014d-494c-b7b9-16f0ad32cae4 admin admin] PATH_INFO: `/v3/users/a768dffa369941038efc57445829d5ee` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:44.416319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270c73b-014d-494c-b7b9-16f0ad32cae4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a768dffa369941038efc57445829d5ee)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:44.418262 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270c73b-014d-494c-b7b9-16f0ad32cae4 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:44.464706 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0270c73b-014d-494c-b7b9-16f0ad32cae4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a768dffa369941038efc57445829d5ee'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:44.473102 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0270c73b-014d-494c-b7b9-16f0ad32cae4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a768dffa369941038efc57445829d5ee'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:44.485594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0270c73b-014d-494c-b7b9-16f0ad32cae4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a768dffa369941038efc57445829d5ee'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:44.493215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0270c73b-014d-494c-b7b9-16f0ad32cae4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a768dffa369941038efc57445829d5ee'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:44.493879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-0270c73b-014d-494c-b7b9-16f0ad32cae4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a768dffa369941038efc57445829d5ee'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:44.500451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1262/2513] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 18:59:44 2022] DELETE /identity/v3/users/a768dffa369941038efc57445829d5ee => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:44.794076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:44.795695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:44.809884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9e3d0-b6de-4b97-b8e5-d7ee732d9692 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:44.850677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:44.879245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015aae3e-d9c4-4739-9fcb-f8b64fbf341c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:44.879896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aae3e-d9c4-4739-9fcb-f8b64fbf341c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:44.880212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aae3e-d9c4-4739-9fcb-f8b64fbf341c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:44.880718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aae3e-d9c4-4739-9fcb-f8b64fbf341c admin admin] PATH_INFO: `/v3/projects/056517beb1d444f5a0406b76794bad98` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:44.882060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015aae3e-d9c4-4739-9fcb-f8b64fbf341c admin admin] RBAC: Authorizing `identity:delete_project(project_id=056517beb1d444f5a0406b76794bad98)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:44.884002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015aae3e-d9c4-4739-9fcb-f8b64fbf341c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:44.936645 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-015aae3e-d9c4-4739-9fcb-f8b64fbf341c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '056517beb1d444f5a0406b76794bad98', 'request_id': 'req-015aae3e-d9c4-4739-9fcb-f8b64fbf341c'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:44.942298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-015aae3e-d9c4-4739-9fcb-f8b64fbf341c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '056517beb1d444f5a0406b76794bad98', 'request_id': 'req-015aae3e-d9c4-4739-9fcb-f8b64fbf341c'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:44.948550 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-015aae3e-d9c4-4739-9fcb-f8b64fbf341c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '056517beb1d444f5a0406b76794bad98', 'request_id': 'req-015aae3e-d9c4-4739-9fcb-f8b64fbf341c'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:44.961265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-015aae3e-d9c4-4739-9fcb-f8b64fbf341c admin admin] The token cache is being invalidate because project 056517beb1d444f5a0406b76794bad98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 18:59:44.961570 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-015aae3e-d9c4-4739-9fcb-f8b64fbf341c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:44.962160 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-015aae3e-d9c4-4739-9fcb-f8b64fbf341c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 18:59:44.963663 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1252/2514] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 18:59:44 2022] DELETE /identity/v3/projects/056517beb1d444f5a0406b76794bad98 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:44.968808 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b4317-84ff-4d6f-b6e2-6b4dbabf4f62 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:44.969095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b4317-84ff-4d6f-b6e2-6b4dbabf4f62 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:44.969368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b4317-84ff-4d6f-b6e2-6b4dbabf4f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:44.997287 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-824b4317-84ff-4d6f-b6e2-6b4dbabf4f62 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:59:44.998499 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-824b4317-84ff-4d6f-b6e2-6b4dbabf4f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:45.018163 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:45.061356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1263/2515] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:59:45.069326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16a5d6-e002-4d27-8480-d1ea039e3b6a None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:45.069617 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16a5d6-e002-4d27-8480-d1ea039e3b6a None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:45.069898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16a5d6-e002-4d27-8480-d1ea039e3b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:45.089287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-4a16a5d6-e002-4d27-8480-d1ea039e3b6a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:59:45.090500 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4a16a5d6-e002-4d27-8480-d1ea039e3b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:45.098456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1253/2516] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:59:45.103210 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b4317-84ff-4d6f-b6e2-6b4dbabf4f62 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:45.113107 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b4317-84ff-4d6f-b6e2-6b4dbabf4f62 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:45.118497 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db8681-8c6d-4438-898d-1cc029d914e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:45.119115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db8681-8c6d-4438-898d-1cc029d914e4 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:45.119396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db8681-8c6d-4438-898d-1cc029d914e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:45.119673 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db8681-8c6d-4438-898d-1cc029d914e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:45.120442 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db8681-8c6d-4438-898d-1cc029d914e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 18:59:45.120751 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db8681-8c6d-4438-898d-1cc029d914e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:45.122366 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db8681-8c6d-4438-898d-1cc029d914e4 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:45.134729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1264/2517] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 18:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 18:59:45.139020 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16a5d6-e002-4d27-8480-d1ea039e3b6a None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:45.148796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16a5d6-e002-4d27-8480-d1ea039e3b6a None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:45.153488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01864-53c5-4d75-b75d-595dea3a172b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:45.154100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01864-53c5-4d75-b75d-595dea3a172b admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:45.154383 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01864-53c5-4d75-b75d-595dea3a172b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:45.154657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01864-53c5-4d75-b75d-595dea3a172b admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:45.155326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e01864-53c5-4d75-b75d-595dea3a172b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:45.157263 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e01864-53c5-4d75-b75d-595dea3a172b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:45.186151 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1254/2518] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 18:59:45 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:59:45.190349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db8681-8c6d-4438-898d-1cc029d914e4 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:45.201036 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db8681-8c6d-4438-898d-1cc029d914e4 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:45.271342 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cde88-12bd-4843-88b6-368dfec3fe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:45.271976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cde88-12bd-4843-88b6-368dfec3fe49 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:45.272262 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cde88-12bd-4843-88b6-368dfec3fe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:45.272575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cde88-12bd-4843-88b6-368dfec3fe49 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:45.273249 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8cde88-12bd-4843-88b6-368dfec3fe49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:45.275191 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8cde88-12bd-4843-88b6-368dfec3fe49 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:45.297076 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1265/2519] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 18:59:45 2022] POST /identity/v3/users => generated 282 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 18:59:45.301446 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01864-53c5-4d75-b75d-595dea3a172b admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:45.313177 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01864-53c5-4d75-b75d-595dea3a172b admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:45.318451 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031029b-5869-4ff7-a541-dc95e69befb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:45.319246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031029b-5869-4ff7-a541-dc95e69befb3 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:45.319424 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031029b-5869-4ff7-a541-dc95e69befb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:45.319699 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031029b-5869-4ff7-a541-dc95e69befb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:45.320447 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031029b-5869-4ff7-a541-dc95e69befb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:45.322048 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031029b-5869-4ff7-a541-dc95e69befb3 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:45.332794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1255/2520] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:59:45.336991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cde88-12bd-4843-88b6-368dfec3fe49 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:45.347103 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cde88-12bd-4843-88b6-368dfec3fe49 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:45.352118 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10883169-46f3-47b2-b1c1-d3e6aa329c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:45.352766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10883169-46f3-47b2-b1c1-d3e6aa329c86 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:45.353046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10883169-46f3-47b2-b1c1-d3e6aa329c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:45.353319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10883169-46f3-47b2-b1c1-d3e6aa329c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:45.354011 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10883169-46f3-47b2-b1c1-d3e6aa329c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:45.355606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10883169-46f3-47b2-b1c1-d3e6aa329c86 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:45.367000 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1266/2521] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 18:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 18:59:45.371415 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031029b-5869-4ff7-a541-dc95e69befb3 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:45.382714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031029b-5869-4ff7-a541-dc95e69befb3 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:45.387452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e0f9e-484f-444f-ab30-813b9991052d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:45.388109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e0f9e-484f-444f-ab30-813b9991052d admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:45.388436 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e0f9e-484f-444f-ab30-813b9991052d admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:45.388717 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e0f9e-484f-444f-ab30-813b9991052d admin admin] PATH_INFO: `/v3/projects/ed25a3bb384d45559522183c27de6e5e/users/273f848bdfdc41c3b7569a43f1a6bdbc/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:45.404377 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e0f9e-484f-444f-ab30-813b9991052d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed25a3bb384d45559522183c27de6e5e, user_id=273f848bdfdc41c3b7569a43f1a6bdbc, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:45.406926 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e0f9e-484f-444f-ab30-813b9991052d admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:45.421812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1256/2522] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 18:59:45 2022] PUT /identity/v3/projects/ed25a3bb384d45559522183c27de6e5e/users/273f848bdfdc41c3b7569a43f1a6bdbc/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 18:59:45.432496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a3f7c-b47b-462c-b4c4-7390de87deb6 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:45.432786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a3f7c-b47b-462c-b4c4-7390de87deb6 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:45.433065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a3f7c-b47b-462c-b4c4-7390de87deb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:45.461232 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-b43a3f7c-b47b-462c-b4c4-7390de87deb6 None None] MFA Rules not processed for user `273f848bdfdc41c3b7569a43f1a6bdbc`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 18:59:45.462444 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b43a3f7c-b47b-462c-b4c4-7390de87deb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:45.529143 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1267/2523] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 18:59:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 18:59:45.541412 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e0f9e-484f-444f-ab30-813b9991052d admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 18:59:45.543108 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-012e0f9e-484f-444f-ab30-813b9991052d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 18:59:45.599645 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e0f9e-484f-444f-ab30-813b9991052d admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 18:59:45.603490 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ab0ef-6c25-4856-97ae-27b537349f37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 18:59:45.604108 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ab0ef-6c25-4856-97ae-27b537349f37 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 18:59:45.604417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ab0ef-6c25-4856-97ae-27b537349f37 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 18:59:45.604697 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ab0ef-6c25-4856-97ae-27b537349f37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 18:59:45.613213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ab0ef-6c25-4856-97ae-27b537349f37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 18:59:45.614637 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ab0ef-6c25-4856-97ae-27b537349f37 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 18:59:45.624526 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1257/2524] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 18:59:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 18:59:55.526148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 18:59:55.620397 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:00:01.955079 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2232f2f-0429-47e2-955a-05633754a293 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:01.955886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2232f2f-0429-47e2-955a-05633754a293 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:01.955886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2232f2f-0429-47e2-955a-05633754a293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:01.976206 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-e2232f2f-0429-47e2-955a-05633754a293 None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:00:01.977495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e2232f2f-0429-47e2-955a-05633754a293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:02.039055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1268/2525] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Aug 3 19:00:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:00:02.086815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ab0ef-6c25-4856-97ae-27b537349f37 service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:02.088218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4a9ab0ef-6c25-4856-97ae-27b537349f37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:02.159851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ab0ef-6c25-4856-97ae-27b537349f37 service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:02.165676 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111a7c1-9343-43a3-9e5e-061bb6cb05c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:02.166306 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111a7c1-9343-43a3-9e5e-061bb6cb05c8 service swift] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:02.166583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111a7c1-9343-43a3-9e5e-061bb6cb05c8 service swift] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:02.166863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111a7c1-9343-43a3-9e5e-061bb6cb05c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:02.175016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111a7c1-9343-43a3-9e5e-061bb6cb05c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:02.176643 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111a7c1-9343-43a3-9e5e-061bb6cb05c8 service swift] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:02.184944 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1258/2526] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 19:00:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 19:00:12.031326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=25 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:00:12.181780 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=32 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:00:14.640898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2232f2f-0429-47e2-955a-05633754a293 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:14.642338 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e2232f2f-0429-47e2-955a-05633754a293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:14.711205 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2232f2f-0429-47e2-955a-05633754a293 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:14.715856 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed316e-1cb2-4e18-b89c-00cbc536dbf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:14.716528 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed316e-1cb2-4e18-b89c-00cbc536dbf2 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:14.716811 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed316e-1cb2-4e18-b89c-00cbc536dbf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:14.717085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed316e-1cb2-4e18-b89c-00cbc536dbf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:14.718863 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-14ed316e-1cb2-4e18-b89c-00cbc536dbf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:14.796198 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed316e-1cb2-4e18-b89c-00cbc536dbf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:14.797823 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed316e-1cb2-4e18-b89c-00cbc536dbf2 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:14.811091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1269/2527] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:00:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:00:16.004204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111a7c1-9343-43a3-9e5e-061bb6cb05c8 service swift] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:16.014563 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111a7c1-9343-43a3-9e5e-061bb6cb05c8 service swift] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:16.018170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337c8e2-e74b-4513-91f4-1b44705ec4c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:16.018779 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337c8e2-e74b-4513-91f4-1b44705ec4c0 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:16.019067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337c8e2-e74b-4513-91f4-1b44705ec4c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:16.019337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337c8e2-e74b-4513-91f4-1b44705ec4c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:16.021115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9337c8e2-e74b-4513-91f4-1b44705ec4c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:16.035322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337c8e2-e74b-4513-91f4-1b44705ec4c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:16.037115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337c8e2-e74b-4513-91f4-1b44705ec4c0 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:16.052529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1259/2528] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:00:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:00:18.272915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1816-ebcb-4cbd-8c64-32b3baffb9f8 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:18.273567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1816-ebcb-4cbd-8c64-32b3baffb9f8 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:18.273567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1816-ebcb-4cbd-8c64-32b3baffb9f8 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:18.275599 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1270/2529] 10.222.0.69 () {54 vars in 921 bytes} [Wed Aug 3 19:00:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 19:00:18.281837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7673b-c488-4d30-bfa5-62effdfe8cb7 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:18.282301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7673b-c488-4d30-bfa5-62effdfe8cb7 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:18.282505 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7673b-c488-4d30-bfa5-62effdfe8cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:18.323785 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-64d7673b-c488-4d30-bfa5-62effdfe8cb7 None None] MFA Rules not processed for user `850a50cfe4394febbafb0908b08888b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:00:18.325043 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-64d7673b-c488-4d30-bfa5-62effdfe8cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:18.334624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1260/2530] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Aug 3 19:00:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:00:18.346809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb1816-ebcb-4cbd-8c64-32b3baffb9f8 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:18.348525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-becb1816-ebcb-4cbd-8c64-32b3baffb9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:18.411925 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb1816-ebcb-4cbd-8c64-32b3baffb9f8 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:18.416417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a024c-177b-4055-90b4-17c07cf4d937 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4799b0c9e5aa47d389121d2183666d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:18.417042 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a024c-177b-4055-90b4-17c07cf4d937 service glance] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:18.417322 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a024c-177b-4055-90b4-17c07cf4d937 service glance] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:18.417596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a024c-177b-4055-90b4-17c07cf4d937 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:18.428122 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a024c-177b-4055-90b4-17c07cf4d937 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:18.429596 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a024c-177b-4055-90b4-17c07cf4d937 service glance] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:18.440232 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1271/2531] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:00:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 19:00:18.510134 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071739e-ca86-46e2-aecc-cd88826aa5b5 None None] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:18.510421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071739e-ca86-46e2-aecc-cd88826aa5b5 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:18.510777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071739e-ca86-46e2-aecc-cd88826aa5b5 None None] PATH_INFO: `/` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:18.512252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1261/2532] 10.222.0.69 () {54 vars in 921 bytes} [Wed Aug 3 19:00:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 19:00:18.516982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1ca63-f7be-41b2-b60e-4ad563c26404 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:18.517320 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1ca63-f7be-41b2-b60e-4ad563c26404 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:18.517601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1ca63-f7be-41b2-b60e-4ad563c26404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:18.538848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-5af1ca63-f7be-41b2-b60e-4ad563c26404 None None] MFA Rules not processed for user `0eb9ad672e5444098d71b7c9a27b72e8`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:00:18.541029 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5af1ca63-f7be-41b2-b60e-4ad563c26404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:18.547890 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1272/2533] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Aug 3 19:00:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:00:18.560954 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071739e-ca86-46e2-aecc-cd88826aa5b5 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:18.562945 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4071739e-ca86-46e2-aecc-cd88826aa5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:18.577253 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071739e-ca86-46e2-aecc-cd88826aa5b5 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:18.580865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8ccda-69ca-4286-a2ca-643fb285f382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:18.581592 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8ccda-69ca-4286-a2ca-643fb285f382 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:18.581931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8ccda-69ca-4286-a2ca-643fb285f382 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:18.582270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8ccda-69ca-4286-a2ca-643fb285f382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:18.589866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8ccda-69ca-4286-a2ca-643fb285f382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:18.591282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8ccda-69ca-4286-a2ca-643fb285f382 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:18.599097 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1262/2534] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:00:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:00:18.691113 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576bc3-a980-4ae6-b5cc-377fe09a3660 None None] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:18.691411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576bc3-a980-4ae6-b5cc-377fe09a3660 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:18.691707 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576bc3-a980-4ae6-b5cc-377fe09a3660 None None] PATH_INFO: `/` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:18.693237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1273/2535] 10.222.0.69 () {54 vars in 921 bytes} [Wed Aug 3 19:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 19:00:18.698950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8e364-f441-445d-bca0-d4dc15751895 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:18.699237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8e364-f441-445d-bca0-d4dc15751895 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:18.699517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8e364-f441-445d-bca0-d4dc15751895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:18.719445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-7ab8e364-f441-445d-bca0-d4dc15751895 None None] MFA Rules not processed for user `850a50cfe4394febbafb0908b08888b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:00:18.720699 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-7ab8e364-f441-445d-bca0-d4dc15751895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:18.733181 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1263/2536] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Aug 3 19:00:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 19:00:18.746369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06576bc3-a980-4ae6-b5cc-377fe09a3660 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:18.758146 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06576bc3-a980-4ae6-b5cc-377fe09a3660 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:18.762709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7168853-2789-4480-8188-00adf16bc515 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:18.763340 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7168853-2789-4480-8188-00adf16bc515 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:18.763682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7168853-2789-4480-8188-00adf16bc515 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:18.763969 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7168853-2789-4480-8188-00adf16bc515 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:18.775302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7168853-2789-4480-8188-00adf16bc515 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:18.776794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7168853-2789-4480-8188-00adf16bc515 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:18.787838 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1274/2537] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:00:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:00:20.574127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ad27-a213-4ce1-95fa-d1ebd4efbbb2 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:20.574610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ad27-a213-4ce1-95fa-d1ebd4efbbb2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:20.574800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ad27-a213-4ce1-95fa-d1ebd4efbbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:20.596700 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-2337ad27-a213-4ce1-95fa-d1ebd4efbbb2 None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:00:20.598060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-2337ad27-a213-4ce1-95fa-d1ebd4efbbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:20.606649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1264/2538] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Aug 3 19:00:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:00:20.620372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7168853-2789-4480-8188-00adf16bc515 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:20.632990 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7168853-2789-4480-8188-00adf16bc515 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:20.639919 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27166a34-7a63-4824-831a-bcc444832267 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:20.640706 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27166a34-7a63-4824-831a-bcc444832267 service swift] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:20.641054 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27166a34-7a63-4824-831a-bcc444832267 service swift] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:20.641387 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27166a34-7a63-4824-831a-bcc444832267 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:20.651611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27166a34-7a63-4824-831a-bcc444832267 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:20.653513 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27166a34-7a63-4824-831a-bcc444832267 service swift] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:20.662969 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1275/2539] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 19:00:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 19:00:30.591543 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=27 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:00:30.659099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=95 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:00:33.804002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337ad27-a213-4ce1-95fa-d1ebd4efbbb2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:33.806498 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-2337ad27-a213-4ce1-95fa-d1ebd4efbbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:33.828372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337ad27-a213-4ce1-95fa-d1ebd4efbbb2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:33.833931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0bda00-97fb-470e-a781-f994bbd55491 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:33.834547 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0bda00-97fb-470e-a781-f994bbd55491 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:33.834834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0bda00-97fb-470e-a781-f994bbd55491 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:33.835111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0bda00-97fb-470e-a781-f994bbd55491 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:33.837438 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-0a0bda00-97fb-470e-a781-f994bbd55491 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:33.919582 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0bda00-97fb-470e-a781-f994bbd55491 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:33.921222 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0bda00-97fb-470e-a781-f994bbd55491 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:33.939923 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1265/2540] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:00:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:00:38.018373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27166a34-7a63-4824-831a-bcc444832267 service swift] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:38.095788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27166a34-7a63-4824-831a-bcc444832267 service swift] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:38.101085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d59286-0912-4a5c-9778-24dca7098460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:38.101724 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d59286-0912-4a5c-9778-24dca7098460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:38.102006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d59286-0912-4a5c-9778-24dca7098460 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:38.102475 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d59286-0912-4a5c-9778-24dca7098460 admin admin] PATH_INFO: `/v3/users/273f848bdfdc41c3b7569a43f1a6bdbc` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:38.103553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d59286-0912-4a5c-9778-24dca7098460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=273f848bdfdc41c3b7569a43f1a6bdbc)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:38.105672 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d59286-0912-4a5c-9778-24dca7098460 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:38.173601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b9d59286-0912-4a5c-9778-24dca7098460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '273f848bdfdc41c3b7569a43f1a6bdbc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:00:38.186252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b9d59286-0912-4a5c-9778-24dca7098460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '273f848bdfdc41c3b7569a43f1a6bdbc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:00:38.200905 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b9d59286-0912-4a5c-9778-24dca7098460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '273f848bdfdc41c3b7569a43f1a6bdbc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:00:38.207846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b9d59286-0912-4a5c-9778-24dca7098460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '273f848bdfdc41c3b7569a43f1a6bdbc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:00:38.208635 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-b9d59286-0912-4a5c-9778-24dca7098460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '273f848bdfdc41c3b7569a43f1a6bdbc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:00:38.216323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1276/2541] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:00:38 2022] DELETE /identity/v3/users/273f848bdfdc41c3b7569a43f1a6bdbc => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:00:38.221246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110e7b6-2b47-42f5-a8b3-f4f06bf4d25b None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:38.221752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110e7b6-2b47-42f5-a8b3-f4f06bf4d25b None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:38.221996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110e7b6-2b47-42f5-a8b3-f4f06bf4d25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:38.241440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-6110e7b6-2b47-42f5-a8b3-f4f06bf4d25b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:00:38.242651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-6110e7b6-2b47-42f5-a8b3-f4f06bf4d25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:38.310730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1266/2542] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:00:38.320681 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d59286-0912-4a5c-9778-24dca7098460 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:38.322982 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-b9d59286-0912-4a5c-9778-24dca7098460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:38.385637 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d59286-0912-4a5c-9778-24dca7098460 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:38.389356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0a4f5-6dfc-44ca-9372-61ac48b8f1b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:38.389981 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0a4f5-6dfc-44ca-9372-61ac48b8f1b6 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:38.390267 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0a4f5-6dfc-44ca-9372-61ac48b8f1b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:38.390545 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0a4f5-6dfc-44ca-9372-61ac48b8f1b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:38.399747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0a4f5-6dfc-44ca-9372-61ac48b8f1b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:38.401251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0a4f5-6dfc-44ca-9372-61ac48b8f1b6 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:38.411192 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1277/2543] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:00:38.705329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110e7b6-2b47-42f5-a8b3-f4f06bf4d25b None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:38.706840 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-6110e7b6-2b47-42f5-a8b3-f4f06bf4d25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:38.720985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110e7b6-2b47-42f5-a8b3-f4f06bf4d25b None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:38.726079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41646665-0c83-4f96-ac1d-bec26039acb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:38.726965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41646665-0c83-4f96-ac1d-bec26039acb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:38.728440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41646665-0c83-4f96-ac1d-bec26039acb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:38.728807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41646665-0c83-4f96-ac1d-bec26039acb8 admin admin] PATH_INFO: `/v3/projects/ed25a3bb384d45559522183c27de6e5e` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:38.730288 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41646665-0c83-4f96-ac1d-bec26039acb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed25a3bb384d45559522183c27de6e5e)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:38.733056 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41646665-0c83-4f96-ac1d-bec26039acb8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:38.796759 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-41646665-0c83-4f96-ac1d-bec26039acb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed25a3bb384d45559522183c27de6e5e', 'request_id': 'req-41646665-0c83-4f96-ac1d-bec26039acb8'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:00:38.802890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-41646665-0c83-4f96-ac1d-bec26039acb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed25a3bb384d45559522183c27de6e5e', 'request_id': 'req-41646665-0c83-4f96-ac1d-bec26039acb8'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:00:38.810397 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-41646665-0c83-4f96-ac1d-bec26039acb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed25a3bb384d45559522183c27de6e5e', 'request_id': 'req-41646665-0c83-4f96-ac1d-bec26039acb8'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:00:38.813119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:00:38.825582 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-41646665-0c83-4f96-ac1d-bec26039acb8 admin admin] The token cache is being invalidate because project ed25a3bb384d45559522183c27de6e5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:00:38.825832 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-41646665-0c83-4f96-ac1d-bec26039acb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:00:38.826792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-41646665-0c83-4f96-ac1d-bec26039acb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:00:38.828272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1267/2544] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:00:38 2022] DELETE /identity/v3/projects/ed25a3bb384d45559522183c27de6e5e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:00:38.833532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ca02b-9eed-4d04-9728-9e929cf3635b None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:38.833747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ca02b-9eed-4d04-9728-9e929cf3635b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:38.834187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ca02b-9eed-4d04-9728-9e929cf3635b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:38.854495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-bc3ca02b-9eed-4d04-9728-9e929cf3635b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:00:38.855685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-bc3ca02b-9eed-4d04-9728-9e929cf3635b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:38.873440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:00:38.917685 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1278/2545] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 19:00:38.925471 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6ded5-d809-4efb-b1b8-b5b52d61f5b3 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:38.925897 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6ded5-d809-4efb-b1b8-b5b52d61f5b3 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:38.926191 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6ded5-d809-4efb-b1b8-b5b52d61f5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:38.946879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-a8b6ded5-d809-4efb-b1b8-b5b52d61f5b3 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:00:38.948142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a8b6ded5-d809-4efb-b1b8-b5b52d61f5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:38.956212 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1268/2546] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:00:38.960460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ca02b-9eed-4d04-9728-9e929cf3635b None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:38.970555 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ca02b-9eed-4d04-9728-9e929cf3635b None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:38.975159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f80f7-5afc-4bb8-9e3e-295104d0e367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:38.975772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f80f7-5afc-4bb8-9e3e-295104d0e367 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:38.976055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f80f7-5afc-4bb8-9e3e-295104d0e367 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:38.976363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f80f7-5afc-4bb8-9e3e-295104d0e367 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:38.977072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f80f7-5afc-4bb8-9e3e-295104d0e367 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:00:38.977927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f80f7-5afc-4bb8-9e3e-295104d0e367 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:38.979664 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f80f7-5afc-4bb8-9e3e-295104d0e367 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:38.991326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1279/2547] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:00:38.996436 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6ded5-d809-4efb-b1b8-b5b52d61f5b3 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:39.006053 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6ded5-d809-4efb-b1b8-b5b52d61f5b3 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:39.010980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bd2bb-867f-49b2-86ef-5be04604a754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:39.011592 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bd2bb-867f-49b2-86ef-5be04604a754 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:39.011905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bd2bb-867f-49b2-86ef-5be04604a754 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:39.012180 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bd2bb-867f-49b2-86ef-5be04604a754 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:39.012895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bd2bb-867f-49b2-86ef-5be04604a754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:39.014801 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bd2bb-867f-49b2-86ef-5be04604a754 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:39.055294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1269/2548] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:00:38 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:00:39.059848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f80f7-5afc-4bb8-9e3e-295104d0e367 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:39.070208 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f80f7-5afc-4bb8-9e3e-295104d0e367 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:39.127236 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b5e27-ea6b-4794-a143-a345944b07c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:39.127857 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b5e27-ea6b-4794-a143-a345944b07c3 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:39.128139 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b5e27-ea6b-4794-a143-a345944b07c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:39.128447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b5e27-ea6b-4794-a143-a345944b07c3 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:39.129110 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b5e27-ea6b-4794-a143-a345944b07c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:39.131044 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b5e27-ea6b-4794-a143-a345944b07c3 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:39.152108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1280/2549] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:00:39 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:00:39.156276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bd2bb-867f-49b2-86ef-5be04604a754 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:39.167673 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bd2bb-867f-49b2-86ef-5be04604a754 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:39.172254 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef69b5-1324-403a-ac77-6947415ad100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:39.172895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef69b5-1324-403a-ac77-6947415ad100 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:39.173173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef69b5-1324-403a-ac77-6947415ad100 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:39.173445 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef69b5-1324-403a-ac77-6947415ad100 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:39.174141 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef69b5-1324-403a-ac77-6947415ad100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:39.175717 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef69b5-1324-403a-ac77-6947415ad100 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:39.186138 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1270/2550] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:00:39.190176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b5e27-ea6b-4794-a143-a345944b07c3 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:39.200683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b5e27-ea6b-4794-a143-a345944b07c3 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:39.205531 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f49c5e-5bd2-4f83-b047-d98cb4e69236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:39.206152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f49c5e-5bd2-4f83-b047-d98cb4e69236 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:39.206502 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f49c5e-5bd2-4f83-b047-d98cb4e69236 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:39.206737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f49c5e-5bd2-4f83-b047-d98cb4e69236 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:39.207409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f49c5e-5bd2-4f83-b047-d98cb4e69236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:39.209065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f49c5e-5bd2-4f83-b047-d98cb4e69236 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:39.220390 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1281/2551] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:00:39.224475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef69b5-1324-403a-ac77-6947415ad100 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:39.235593 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef69b5-1324-403a-ac77-6947415ad100 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:39.240232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3840f-45f9-453e-802a-19bc1dfadade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:39.240934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3840f-45f9-453e-802a-19bc1dfadade admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:39.241302 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3840f-45f9-453e-802a-19bc1dfadade admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:39.241584 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3840f-45f9-453e-802a-19bc1dfadade admin admin] PATH_INFO: `/v3/projects/c00e10f7a413422dba1ab74ef86e0946/users/f84fe9ee2c5d48868d73ae651e253f9e/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:39.256612 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3840f-45f9-453e-802a-19bc1dfadade admin admin] RBAC: Authorizing `identity:create_grant(project_id=c00e10f7a413422dba1ab74ef86e0946, user_id=f84fe9ee2c5d48868d73ae651e253f9e, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:39.259476 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3840f-45f9-453e-802a-19bc1dfadade admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:39.272233 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1271/2552] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:00:39 2022] PUT /identity/v3/projects/c00e10f7a413422dba1ab74ef86e0946/users/f84fe9ee2c5d48868d73ae651e253f9e/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:00:39.277764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c703-6c4e-4cf5-8702-9439a49ff734 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:39.278076 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c703-6c4e-4cf5-8702-9439a49ff734 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:39.278352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c703-6c4e-4cf5-8702-9439a49ff734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:39.300032 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-5cc7c703-6c4e-4cf5-8702-9439a49ff734 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:00:39.301285 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5cc7c703-6c4e-4cf5-8702-9439a49ff734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:39.360780 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1282/2553] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:00:39.371097 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3840f-45f9-453e-802a-19bc1dfadade admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:39.373155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-99b3840f-45f9-453e-802a-19bc1dfadade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:39.437917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3840f-45f9-453e-802a-19bc1dfadade admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:39.441792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d7567-fbc5-43e9-adf2-2de1e656f02f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:39.442453 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d7567-fbc5-43e9-adf2-2de1e656f02f service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:39.442733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d7567-fbc5-43e9-adf2-2de1e656f02f service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:39.443038 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d7567-fbc5-43e9-adf2-2de1e656f02f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:39.453160 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d7567-fbc5-43e9-adf2-2de1e656f02f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:39.454632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d7567-fbc5-43e9-adf2-2de1e656f02f service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:39.464582 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1272/2554] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:00:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:00:40.463473 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7c703-6c4e-4cf5-8702-9439a49ff734 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:40.474061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7c703-6c4e-4cf5-8702-9439a49ff734 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:40.479222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fc93a-97bd-4e16-a53b-7da50d25138e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:40.479992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fc93a-97bd-4e16-a53b-7da50d25138e service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:40.480335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fc93a-97bd-4e16-a53b-7da50d25138e service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:40.480609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fc93a-97bd-4e16-a53b-7da50d25138e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:40.482760 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-cd2fc93a-97bd-4e16-a53b-7da50d25138e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:40.493143 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fc93a-97bd-4e16-a53b-7da50d25138e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:40.494633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fc93a-97bd-4e16-a53b-7da50d25138e service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:40.504515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1283/2555] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:00:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:00:42.184664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8771bb2-ef99-45ee-a779-6a2853e9dbd3 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:42.185641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8771bb2-ef99-45ee-a779-6a2853e9dbd3 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:42.185641 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8771bb2-ef99-45ee-a779-6a2853e9dbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:42.253013 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-f8771bb2-ef99-45ee-a779-6a2853e9dbd3 None None] MFA Rules not processed for user `6cebfb2bb96942c9b64720d9b8d9c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:00:42.254268 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f8771bb2-ef99-45ee-a779-6a2853e9dbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:42.339787 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:00:42.367629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1273/2556] 10.222.0.69 () {58 vars in 967 bytes} [Wed Aug 3 19:00:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:00:42.383166 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fc93a-97bd-4e16-a53b-7da50d25138e service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:42.387788 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-cd2fc93a-97bd-4e16-a53b-7da50d25138e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:42.424520 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:00:42.546214 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fc93a-97bd-4e16-a53b-7da50d25138e service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:42.553825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c33ab-d752-4bc5-8b1b-e97cb310442f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96ed059287274ca392c6abb893551094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:42.555133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c33ab-d752-4bc5-8b1b-e97cb310442f service swift] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:42.555133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c33ab-d752-4bc5-8b1b-e97cb310442f service swift] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:42.556005 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c33ab-d752-4bc5-8b1b-e97cb310442f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:42.571361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c33ab-d752-4bc5-8b1b-e97cb310442f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:42.573129 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c33ab-d752-4bc5-8b1b-e97cb310442f service swift] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:42.593174 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1284/2557] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Aug 3 19:00:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 19:00:43.903118 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3f0e1-d93e-4147-962d-2abf1a004e82 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:43.903604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3f0e1-d93e-4147-962d-2abf1a004e82 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:43.903785 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3f0e1-d93e-4147-962d-2abf1a004e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:43.935690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-0fe3f0e1-d93e-4147-962d-2abf1a004e82 None None] MFA Rules not processed for user `f84fe9ee2c5d48868d73ae651e253f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:00:43.936948 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-0fe3f0e1-d93e-4147-962d-2abf1a004e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:44.012987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1274/2558] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:00:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:00:44.017602 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c33ab-d752-4bc5-8b1b-e97cb310442f service swift] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:44.050250 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c33ab-d752-4bc5-8b1b-e97cb310442f service swift] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:44.054985 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13edff1-4654-4e53-b11b-636f13c73d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:44.055677 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13edff1-4654-4e53-b11b-636f13c73d25 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:44.055983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13edff1-4654-4e53-b11b-636f13c73d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:44.056259 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13edff1-4654-4e53-b11b-636f13c73d25 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:44.056962 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13edff1-4654-4e53-b11b-636f13c73d25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:44.058884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13edff1-4654-4e53-b11b-636f13c73d25 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:44.101867 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1285/2559] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:00:44 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 19:00:44.106248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3f0e1-d93e-4147-962d-2abf1a004e82 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:44.117589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3f0e1-d93e-4147-962d-2abf1a004e82 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:44.179449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45d488-013f-4e70-802e-aea3429ae73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:44.180080 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45d488-013f-4e70-802e-aea3429ae73a admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:44.180471 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45d488-013f-4e70-802e-aea3429ae73a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:44.180802 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45d488-013f-4e70-802e-aea3429ae73a admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:44.181449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45d488-013f-4e70-802e-aea3429ae73a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:44.183383 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45d488-013f-4e70-802e-aea3429ae73a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:44.213689 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1275/2560] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:00:44 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:00:44.214759 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13edff1-4654-4e53-b11b-636f13c73d25 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:44.226505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13edff1-4654-4e53-b11b-636f13c73d25 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:44.231622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51710d0c-ddc1-4801-8a1b-3c848a31bf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:44.232262 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51710d0c-ddc1-4801-8a1b-3c848a31bf42 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:44.232593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51710d0c-ddc1-4801-8a1b-3c848a31bf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:44.233033 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51710d0c-ddc1-4801-8a1b-3c848a31bf42 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:44.233906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51710d0c-ddc1-4801-8a1b-3c848a31bf42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:44.235537 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51710d0c-ddc1-4801-8a1b-3c848a31bf42 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:44.247160 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1286/2561] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:00:44.251699 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45d488-013f-4e70-802e-aea3429ae73a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:44.263205 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45d488-013f-4e70-802e-aea3429ae73a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:44.267928 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb93bf-9ad1-4b8f-9211-2a71854e9e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:44.268647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb93bf-9ad1-4b8f-9211-2a71854e9e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:44.268932 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb93bf-9ad1-4b8f-9211-2a71854e9e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:44.269229 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb93bf-9ad1-4b8f-9211-2a71854e9e93 admin admin] PATH_INFO: `/v3/projects/a16ee75d29f74109aee3b1be4042ac7a/users/7b51e4f2a1664c94bcc57aed111249fd/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:44.286318 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb93bf-9ad1-4b8f-9211-2a71854e9e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a16ee75d29f74109aee3b1be4042ac7a, user_id=7b51e4f2a1664c94bcc57aed111249fd, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:44.289361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb93bf-9ad1-4b8f-9211-2a71854e9e93 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:44.304127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1276/2562] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:00:44 2022] PUT /identity/v3/projects/a16ee75d29f74109aee3b1be4042ac7a/users/7b51e4f2a1664c94bcc57aed111249fd/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:00:44.308247 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51710d0c-ddc1-4801-8a1b-3c848a31bf42 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:44.319127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51710d0c-ddc1-4801-8a1b-3c848a31bf42 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:44.382717 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260fb7c-c8cf-4707-a35a-1a1e68f65d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:44.383375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260fb7c-c8cf-4707-a35a-1a1e68f65d08 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:44.384989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260fb7c-c8cf-4707-a35a-1a1e68f65d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:44.385263 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260fb7c-c8cf-4707-a35a-1a1e68f65d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:44.385964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260fb7c-c8cf-4707-a35a-1a1e68f65d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:44.387575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260fb7c-c8cf-4707-a35a-1a1e68f65d08 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:44.400196 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1287/2563] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 19:00:44.404574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb93bf-9ad1-4b8f-9211-2a71854e9e93 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:44.416784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb93bf-9ad1-4b8f-9211-2a71854e9e93 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:44.421667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55042c8-b93e-42de-89af-4ab8b5986f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:44.422350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55042c8-b93e-42de-89af-4ab8b5986f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:44.422633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55042c8-b93e-42de-89af-4ab8b5986f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:44.422914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55042c8-b93e-42de-89af-4ab8b5986f69 admin admin] PATH_INFO: `/v3/projects/a16ee75d29f74109aee3b1be4042ac7a/users/7b51e4f2a1664c94bcc57aed111249fd/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:44.424909 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55042c8-b93e-42de-89af-4ab8b5986f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a16ee75d29f74109aee3b1be4042ac7a, user_id=7b51e4f2a1664c94bcc57aed111249fd, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:44.427561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55042c8-b93e-42de-89af-4ab8b5986f69 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:44.439510 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1277/2564] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:00:44 2022] PUT /identity/v3/projects/a16ee75d29f74109aee3b1be4042ac7a/users/7b51e4f2a1664c94bcc57aed111249fd/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:00:48.943801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdb7a9-10fc-46c7-b737-3e5959d7ce09 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:48.944251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdb7a9-10fc-46c7-b737-3e5959d7ce09 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:48.944508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdb7a9-10fc-46c7-b737-3e5959d7ce09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:48.983639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-eafdb7a9-10fc-46c7-b737-3e5959d7ce09 None None] MFA Rules not processed for user `7b51e4f2a1664c94bcc57aed111249fd`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:00:48.984228 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-eafdb7a9-10fc-46c7-b737-3e5959d7ce09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:49.036763 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:00:49.075322 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1288/2565] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:00:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 19:00:49.090348 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55042c8-b93e-42de-89af-4ab8b5986f69 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:49.092556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c55042c8-b93e-42de-89af-4ab8b5986f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:49.169791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:00:49.240530 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55042c8-b93e-42de-89af-4ab8b5986f69 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:49.247762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041f69e-237d-4e71-9d83-f3537254b0e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:49.248618 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041f69e-237d-4e71-9d83-f3537254b0e0 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:49.248998 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041f69e-237d-4e71-9d83-f3537254b0e0 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:49.249369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041f69e-237d-4e71-9d83-f3537254b0e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:49.328272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041f69e-237d-4e71-9d83-f3537254b0e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:49.330383 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041f69e-237d-4e71-9d83-f3537254b0e0 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:49.341343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1278/2566] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:00:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:00:50.413782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe15f0-42a9-47b3-92c9-9ff64f2e591c None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:50.414294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe15f0-42a9-47b3-92c9-9ff64f2e591c None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:50.414390 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe15f0-42a9-47b3-92c9-9ff64f2e591c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:50.434859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-21fe15f0-42a9-47b3-92c9-9ff64f2e591c None None] MFA Rules not processed for user `f84fe9ee2c5d48868d73ae651e253f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:00:50.436628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-21fe15f0-42a9-47b3-92c9-9ff64f2e591c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:50.445315 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1289/2567] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:00:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:00:51.315315 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041f69e-237d-4e71-9d83-f3537254b0e0 service nova] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:00:51.316790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f041f69e-237d-4e71-9d83-f3537254b0e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:51.338387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041f69e-237d-4e71-9d83-f3537254b0e0 service nova] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:00:51.344506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a552a8-791a-4ef4-bfbc-665a1c071889 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:00:51.345162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a552a8-791a-4ef4-bfbc-665a1c071889 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:00:51.345474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a552a8-791a-4ef4-bfbc-665a1c071889 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:00:51.345801 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a552a8-791a-4ef4-bfbc-665a1c071889 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:00:51.347922 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-06a552a8-791a-4ef4-bfbc-665a1c071889 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:00:51.439546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a552a8-791a-4ef4-bfbc-665a1c071889 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:00:51.441185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a552a8-791a-4ef4-bfbc-665a1c071889 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:00:51.465656 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1279/2568] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:00:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:01:00.428894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=15 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:01:01.461215 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=80 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:01:17.113500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe15f0-42a9-47b3-92c9-9ff64f2e591c None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:17.113500 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-21fe15f0-42a9-47b3-92c9-9ff64f2e591c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:17.202123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe15f0-42a9-47b3-92c9-9ff64f2e591c None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:17.207515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6d4e8-80e2-4a2b-8481-b7a0ee70ae84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:17.208177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6d4e8-80e2-4a2b-8481-b7a0ee70ae84 service placement] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:17.208485 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6d4e8-80e2-4a2b-8481-b7a0ee70ae84 service placement] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:17.208781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6d4e8-80e2-4a2b-8481-b7a0ee70ae84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:17.210507 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-59c6d4e8-80e2-4a2b-8481-b7a0ee70ae84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:17.223354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6d4e8-80e2-4a2b-8481-b7a0ee70ae84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:17.224991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6d4e8-80e2-4a2b-8481-b7a0ee70ae84 service placement] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:17.236529 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1290/2569] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:01:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:01:21.774677 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a552a8-791a-4ef4-bfbc-665a1c071889 service nova] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:21.793506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a552a8-791a-4ef4-bfbc-665a1c071889 service nova] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:21.818786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199ef89-5ea9-4f53-8da5-147ffab29ed7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:21.819686 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199ef89-5ea9-4f53-8da5-147ffab29ed7 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:21.820179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199ef89-5ea9-4f53-8da5-147ffab29ed7 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:21.820781 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199ef89-5ea9-4f53-8da5-147ffab29ed7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:21.823898 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a199ef89-5ea9-4f53-8da5-147ffab29ed7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:21.843644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199ef89-5ea9-4f53-8da5-147ffab29ed7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:21.845600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199ef89-5ea9-4f53-8da5-147ffab29ed7 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:21.858767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1280/2570] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:01:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:01:27.232057 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=32 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:01:27.631688 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6d4e8-80e2-4a2b-8481-b7a0ee70ae84 service placement] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:27.690594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6d4e8-80e2-4a2b-8481-b7a0ee70ae84 service placement] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:27.694188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f09ab0-1f44-4665-a4b2-e53743dba85a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:27.694814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f09ab0-1f44-4665-a4b2-e53743dba85a service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:27.695094 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f09ab0-1f44-4665-a4b2-e53743dba85a service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:27.695367 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f09ab0-1f44-4665-a4b2-e53743dba85a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:27.697169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e9f09ab0-1f44-4665-a4b2-e53743dba85a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:27.762452 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f09ab0-1f44-4665-a4b2-e53743dba85a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:27.763907 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f09ab0-1f44-4665-a4b2-e53743dba85a service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:27.773535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1291/2571] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:01:30.840480 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199ef89-5ea9-4f53-8da5-147ffab29ed7 service nova] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:30.852727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199ef89-5ea9-4f53-8da5-147ffab29ed7 service nova] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:30.858856 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425abdfc-40e0-4774-a9b8-724bc3b12c01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:30.859488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425abdfc-40e0-4774-a9b8-724bc3b12c01 service placement] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:30.859769 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425abdfc-40e0-4774-a9b8-724bc3b12c01 service placement] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:30.860042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425abdfc-40e0-4774-a9b8-724bc3b12c01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:30.861975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-425abdfc-40e0-4774-a9b8-724bc3b12c01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:30.875707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425abdfc-40e0-4774-a9b8-724bc3b12c01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:30.877330 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425abdfc-40e0-4774-a9b8-724bc3b12c01 service placement] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:30.890461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1281/2572] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:01:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:01:30.902353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f09ab0-1f44-4665-a4b2-e53743dba85a service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:30.908024 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e9f09ab0-1f44-4665-a4b2-e53743dba85a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:30.923915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f09ab0-1f44-4665-a4b2-e53743dba85a service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:30.928698 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c94bc8-c390-4d57-8b90-938a36ed67c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:30.933114 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c94bc8-c390-4d57-8b90-938a36ed67c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:30.933405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c94bc8-c390-4d57-8b90-938a36ed67c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:30.934228 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c94bc8-c390-4d57-8b90-938a36ed67c3 admin admin] PATH_INFO: `/v3/users/13ab977f852a41e898af1ddfb67bad69` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:30.937101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c94bc8-c390-4d57-8b90-938a36ed67c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ab977f852a41e898af1ddfb67bad69)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:30.939157 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c94bc8-c390-4d57-8b90-938a36ed67c3 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:30.994800 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-84c94bc8-c390-4d57-8b90-938a36ed67c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ab977f852a41e898af1ddfb67bad69'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:01:31.003182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-84c94bc8-c390-4d57-8b90-938a36ed67c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ab977f852a41e898af1ddfb67bad69'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:01:31.016392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-84c94bc8-c390-4d57-8b90-938a36ed67c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ab977f852a41e898af1ddfb67bad69'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:01:31.023254 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-84c94bc8-c390-4d57-8b90-938a36ed67c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ab977f852a41e898af1ddfb67bad69'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:01:31.023933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-84c94bc8-c390-4d57-8b90-938a36ed67c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ab977f852a41e898af1ddfb67bad69'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:01:31.030274 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1292/2573] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:01:30 2022] DELETE /identity/v3/users/13ab977f852a41e898af1ddfb67bad69 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:01:31.213792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425abdfc-40e0-4774-a9b8-724bc3b12c01 service placement] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:31.227232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425abdfc-40e0-4774-a9b8-724bc3b12c01 service placement] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:31.290890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece92e1d-4060-4b2f-81ad-2b981c67c523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:31.291611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece92e1d-4060-4b2f-81ad-2b981c67c523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:31.291960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece92e1d-4060-4b2f-81ad-2b981c67c523 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:31.292326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece92e1d-4060-4b2f-81ad-2b981c67c523 admin admin] PATH_INFO: `/v3/projects/77d5ff41add94950b7422c6de3e36257` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:31.294987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece92e1d-4060-4b2f-81ad-2b981c67c523 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d5ff41add94950b7422c6de3e36257)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:31.297095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece92e1d-4060-4b2f-81ad-2b981c67c523 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:31.353904 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ece92e1d-4060-4b2f-81ad-2b981c67c523 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d5ff41add94950b7422c6de3e36257', 'request_id': 'req-ece92e1d-4060-4b2f-81ad-2b981c67c523'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:01:31.360299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ece92e1d-4060-4b2f-81ad-2b981c67c523 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d5ff41add94950b7422c6de3e36257', 'request_id': 'req-ece92e1d-4060-4b2f-81ad-2b981c67c523'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:01:31.369535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ece92e1d-4060-4b2f-81ad-2b981c67c523 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d5ff41add94950b7422c6de3e36257', 'request_id': 'req-ece92e1d-4060-4b2f-81ad-2b981c67c523'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:01:31.387590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ece92e1d-4060-4b2f-81ad-2b981c67c523 admin admin] The token cache is being invalidate because project 77d5ff41add94950b7422c6de3e36257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:01:31.387871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ece92e1d-4060-4b2f-81ad-2b981c67c523 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:01:31.388572 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ece92e1d-4060-4b2f-81ad-2b981c67c523 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:01:31.390214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1282/2574] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:01:31 2022] DELETE /identity/v3/projects/77d5ff41add94950b7422c6de3e36257 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:01:31.395285 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a288af-da3a-481d-bd40-cf40ebf62947 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:31.395573 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a288af-da3a-481d-bd40-cf40ebf62947 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:31.395852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a288af-da3a-481d-bd40-cf40ebf62947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:31.416651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-a6a288af-da3a-481d-bd40-cf40ebf62947 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:01:31.418375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a6a288af-da3a-481d-bd40-cf40ebf62947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:31.440330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:01:31.486987 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1293/2575] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:01:31.494985 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3897754-0de1-4f03-817c-2f3659325d81 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:31.495317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3897754-0de1-4f03-817c-2f3659325d81 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:31.495640 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3897754-0de1-4f03-817c-2f3659325d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:31.516589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-a3897754-0de1-4f03-817c-2f3659325d81 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:01:31.517806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a3897754-0de1-4f03-817c-2f3659325d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:31.525803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1283/2576] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:01:31.531010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a288af-da3a-481d-bd40-cf40ebf62947 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:31.541011 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a288af-da3a-481d-bd40-cf40ebf62947 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:31.546621 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514bbaa-1d80-46c3-a4b0-810959897b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:31.547253 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514bbaa-1d80-46c3-a4b0-810959897b4c admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:31.547536 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514bbaa-1d80-46c3-a4b0-810959897b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:31.547814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514bbaa-1d80-46c3-a4b0-810959897b4c admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:31.548586 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4514bbaa-1d80-46c3-a4b0-810959897b4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:01:31.548898 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4514bbaa-1d80-46c3-a4b0-810959897b4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:31.550528 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4514bbaa-1d80-46c3-a4b0-810959897b4c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:31.562906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1294/2577] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:01:31.567640 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3897754-0de1-4f03-817c-2f3659325d81 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:31.577605 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3897754-0de1-4f03-817c-2f3659325d81 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:31.582350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d208d66-1279-4f70-8504-69bc29f3d774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:31.582981 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d208d66-1279-4f70-8504-69bc29f3d774 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:31.583265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d208d66-1279-4f70-8504-69bc29f3d774 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:31.583543 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d208d66-1279-4f70-8504-69bc29f3d774 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:31.584217 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d208d66-1279-4f70-8504-69bc29f3d774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:31.586172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d208d66-1279-4f70-8504-69bc29f3d774 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:31.619693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1284/2578] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:01:31 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:01:31.621555 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514bbaa-1d80-46c3-a4b0-810959897b4c admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:31.634431 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514bbaa-1d80-46c3-a4b0-810959897b4c admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:31.693865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f0f05-4806-4642-a562-fccf83159387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:31.694495 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f0f05-4806-4642-a562-fccf83159387 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:31.694776 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f0f05-4806-4642-a562-fccf83159387 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:31.695052 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f0f05-4806-4642-a562-fccf83159387 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:31.695720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f0f05-4806-4642-a562-fccf83159387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:31.697709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f0f05-4806-4642-a562-fccf83159387 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:31.721285 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1295/2579] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:01:31 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:01:31.725562 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d208d66-1279-4f70-8504-69bc29f3d774 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:31.737224 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d208d66-1279-4f70-8504-69bc29f3d774 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:31.742090 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ede2a0-c47f-46d7-b4ea-17cbbc33e67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:31.742710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ede2a0-c47f-46d7-b4ea-17cbbc33e67a admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:31.742994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ede2a0-c47f-46d7-b4ea-17cbbc33e67a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:31.743273 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ede2a0-c47f-46d7-b4ea-17cbbc33e67a admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:31.743971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ede2a0-c47f-46d7-b4ea-17cbbc33e67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:31.745623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ede2a0-c47f-46d7-b4ea-17cbbc33e67a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:31.757236 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1285/2580] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 19:01:31.761396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f0f05-4806-4642-a562-fccf83159387 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:31.772282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f0f05-4806-4642-a562-fccf83159387 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:31.779840 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbb0c9-d924-4db4-88e8-3d25f646e4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:31.780525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbb0c9-d924-4db4-88e8-3d25f646e4ba admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:31.780808 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbb0c9-d924-4db4-88e8-3d25f646e4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:31.781105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbb0c9-d924-4db4-88e8-3d25f646e4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:31.781849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbb0c9-d924-4db4-88e8-3d25f646e4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:31.783580 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbb0c9-d924-4db4-88e8-3d25f646e4ba admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:31.795861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1296/2581] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 19:01:31.800461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ede2a0-c47f-46d7-b4ea-17cbbc33e67a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:31.811448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ede2a0-c47f-46d7-b4ea-17cbbc33e67a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:31.816193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aaab42-e0c1-4c5d-84f3-4ccb21b4a050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:31.816943 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaab42-e0c1-4c5d-84f3-4ccb21b4a050 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:31.817261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaab42-e0c1-4c5d-84f3-4ccb21b4a050 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:31.817540 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaab42-e0c1-4c5d-84f3-4ccb21b4a050 admin admin] PATH_INFO: `/v3/projects/6a6eee7038a04ad988d383473d0113a4/users/cd2bc7050bb9413c8fddd9c7864f537a/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:31.833103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aaab42-e0c1-4c5d-84f3-4ccb21b4a050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a6eee7038a04ad988d383473d0113a4, user_id=cd2bc7050bb9413c8fddd9c7864f537a, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:31.835808 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aaab42-e0c1-4c5d-84f3-4ccb21b4a050 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:31.847405 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:01:31.853244 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1286/2582] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:01:31 2022] PUT /identity/v3/projects/6a6eee7038a04ad988d383473d0113a4/users/cd2bc7050bb9413c8fddd9c7864f537a/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 19:01:31.859265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707efbd-0a64-4964-a3ae-1eb06e35d583 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:31.859586 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707efbd-0a64-4964-a3ae-1eb06e35d583 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:31.859865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707efbd-0a64-4964-a3ae-1eb06e35d583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:31.880927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-f707efbd-0a64-4964-a3ae-1eb06e35d583 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:01:31.882842 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f707efbd-0a64-4964-a3ae-1eb06e35d583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:31.951600 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1297/2583] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:01:31.961275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aaab42-e0c1-4c5d-84f3-4ccb21b4a050 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:31.962960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-96aaab42-e0c1-4c5d-84f3-4ccb21b4a050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:32.029500 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aaab42-e0c1-4c5d-84f3-4ccb21b4a050 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:32.034293 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a052295-e3d6-4c24-86df-6629ae9d75f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:32.034924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a052295-e3d6-4c24-86df-6629ae9d75f7 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:32.035206 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a052295-e3d6-4c24-86df-6629ae9d75f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:32.035475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a052295-e3d6-4c24-86df-6629ae9d75f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:32.049175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a052295-e3d6-4c24-86df-6629ae9d75f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:32.050625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a052295-e3d6-4c24-86df-6629ae9d75f7 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:32.060797 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1287/2584] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:01:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:01:34.695535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707efbd-0a64-4964-a3ae-1eb06e35d583 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:34.709796 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f707efbd-0a64-4964-a3ae-1eb06e35d583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:34.785992 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:01:34.998572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707efbd-0a64-4964-a3ae-1eb06e35d583 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:34.998572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cf645-8bc2-4353-bba1-b8082af91097 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:34.998572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cf645-8bc2-4353-bba1-b8082af91097 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:34.998572 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cf645-8bc2-4353-bba1-b8082af91097 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:35.002115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cf645-8bc2-4353-bba1-b8082af91097 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:35.002115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-971cf645-8bc2-4353-bba1-b8082af91097 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:35.002115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cf645-8bc2-4353-bba1-b8082af91097 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:35.002115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cf645-8bc2-4353-bba1-b8082af91097 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:35.002115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1298/2585] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:01:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:01:36.878668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3935a-d1b3-43bd-a955-ea4cc45eb3de None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:36.879226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3935a-d1b3-43bd-a955-ea4cc45eb3de None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:36.879472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3935a-d1b3-43bd-a955-ea4cc45eb3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:36.914981 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-4ec3935a-d1b3-43bd-a955-ea4cc45eb3de None None] MFA Rules not processed for user `cd2bc7050bb9413c8fddd9c7864f537a`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:01:36.916755 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4ec3935a-d1b3-43bd-a955-ea4cc45eb3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:37.000974 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1288/2586] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:01:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:01:37.016585 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cf645-8bc2-4353-bba1-b8082af91097 service nova] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:37.018786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-971cf645-8bc2-4353-bba1-b8082af91097 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:37.036506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cf645-8bc2-4353-bba1-b8082af91097 service nova] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:37.042640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443daf7c-d28a-4bd2-96b5-7be2789aafb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:37.044126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443daf7c-d28a-4bd2-96b5-7be2789aafb0 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:37.044273 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443daf7c-d28a-4bd2-96b5-7be2789aafb0 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:37.044754 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443daf7c-d28a-4bd2-96b5-7be2789aafb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:37.053860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443daf7c-d28a-4bd2-96b5-7be2789aafb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:37.055603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443daf7c-d28a-4bd2-96b5-7be2789aafb0 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:37.065592 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1299/2587] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:01:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:01:38.497880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3935a-d1b3-43bd-a955-ea4cc45eb3de None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:38.518097 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3935a-d1b3-43bd-a955-ea4cc45eb3de None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:38.524386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af4f68-1946-4064-b436-918f7c277296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:38.525964 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af4f68-1946-4064-b436-918f7c277296 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:38.526421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af4f68-1946-4064-b436-918f7c277296 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:38.526914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af4f68-1946-4064-b436-918f7c277296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:38.530377 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-21af4f68-1946-4064-b436-918f7c277296 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:38.575779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d4543-8647-4aab-a213-558c913a24a9 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:38.576699 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d4543-8647-4aab-a213-558c913a24a9 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:38.576699 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d4543-8647-4aab-a213-558c913a24a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:38.596707 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-fb1d4543-8647-4aab-a213-558c913a24a9 None None] MFA Rules not processed for user `cd2bc7050bb9413c8fddd9c7864f537a`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:01:38.598155 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-fb1d4543-8647-4aab-a213-558c913a24a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:38.606080 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1300/2588] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:01:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 19:01:38.625994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af4f68-1946-4064-b436-918f7c277296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:38.628886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af4f68-1946-4064-b436-918f7c277296 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:38.643290 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1289/2589] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:01:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:01:48.590997 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=30 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:01:48.638417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=88 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:01:58.412901 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d4543-8647-4aab-a213-558c913a24a9 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:01:58.423386 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d4543-8647-4aab-a213-558c913a24a9 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:01:58.427055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a329a0-0bca-4e62-9505-86b01e1f1f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:01:58.427783 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a329a0-0bca-4e62-9505-86b01e1f1f96 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:01:58.428072 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a329a0-0bca-4e62-9505-86b01e1f1f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:01:58.428382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a329a0-0bca-4e62-9505-86b01e1f1f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:01:58.430224 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e9a329a0-0bca-4e62-9505-86b01e1f1f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:01:58.444010 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a329a0-0bca-4e62-9505-86b01e1f1f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:01:58.445503 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a329a0-0bca-4e62-9505-86b01e1f1f96 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:01:58.456691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1301/2590] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:02:01.639408 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af4f68-1946-4064-b436-918f7c277296 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:01.641153 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-21af4f68-1946-4064-b436-918f7c277296 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:01.660521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af4f68-1946-4064-b436-918f7c277296 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:01.665084 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5d195-6adc-4387-857c-028921a83d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:01.665729 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5d195-6adc-4387-857c-028921a83d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:01.666012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5d195-6adc-4387-857c-028921a83d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:01.666288 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5d195-6adc-4387-857c-028921a83d50 admin admin] PATH_INFO: `/v3/users/1aa1d56d9b154e32aa1c136b577d49f4` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:01.667460 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5d195-6adc-4387-857c-028921a83d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aa1d56d9b154e32aa1c136b577d49f4)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:01.670304 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5d195-6adc-4387-857c-028921a83d50 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:01.717706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ebe5d195-6adc-4387-857c-028921a83d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aa1d56d9b154e32aa1c136b577d49f4'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:01.727646 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ebe5d195-6adc-4387-857c-028921a83d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aa1d56d9b154e32aa1c136b577d49f4'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:01.741297 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ebe5d195-6adc-4387-857c-028921a83d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aa1d56d9b154e32aa1c136b577d49f4'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:01.748148 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ebe5d195-6adc-4387-857c-028921a83d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aa1d56d9b154e32aa1c136b577d49f4'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:01.748809 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-ebe5d195-6adc-4387-857c-028921a83d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aa1d56d9b154e32aa1c136b577d49f4'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:01.755348 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1290/2591] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:02:01 2022] DELETE /identity/v3/users/1aa1d56d9b154e32aa1c136b577d49f4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 19:02:01.934687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a329a0-0bca-4e62-9505-86b01e1f1f96 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:01.945164 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a329a0-0bca-4e62-9505-86b01e1f1f96 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:02.005733 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88e567-c72e-455e-b3d5-a86f6339039e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:02.006401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88e567-c72e-455e-b3d5-a86f6339039e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:02.006687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88e567-c72e-455e-b3d5-a86f6339039e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:02.006965 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88e567-c72e-455e-b3d5-a86f6339039e admin admin] PATH_INFO: `/v3/projects/e125626c9566484b837b582ba7096598` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:02.008133 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88e567-c72e-455e-b3d5-a86f6339039e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e125626c9566484b837b582ba7096598)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:02.010077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88e567-c72e-455e-b3d5-a86f6339039e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:02.057497 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8d88e567-c72e-455e-b3d5-a86f6339039e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e125626c9566484b837b582ba7096598', 'request_id': 'req-8d88e567-c72e-455e-b3d5-a86f6339039e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:02.063117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8d88e567-c72e-455e-b3d5-a86f6339039e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e125626c9566484b837b582ba7096598', 'request_id': 'req-8d88e567-c72e-455e-b3d5-a86f6339039e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:02.070775 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8d88e567-c72e-455e-b3d5-a86f6339039e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e125626c9566484b837b582ba7096598', 'request_id': 'req-8d88e567-c72e-455e-b3d5-a86f6339039e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:02.082787 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8d88e567-c72e-455e-b3d5-a86f6339039e admin admin] The token cache is being invalidate because project e125626c9566484b837b582ba7096598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:02:02.083081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8d88e567-c72e-455e-b3d5-a86f6339039e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:02.083714 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8d88e567-c72e-455e-b3d5-a86f6339039e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:02.085258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1302/2592] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:02:01 2022] DELETE /identity/v3/projects/e125626c9566484b837b582ba7096598 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:02.090740 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c942e-f7c3-4457-8670-d9319b5d1fef None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:02.091179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c942e-f7c3-4457-8670-d9319b5d1fef None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:02.091464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c942e-f7c3-4457-8670-d9319b5d1fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:02.111015 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-a33c942e-f7c3-4457-8670-d9319b5d1fef None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:02.112228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a33c942e-f7c3-4457-8670-d9319b5d1fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:02.172051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1291/2593] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:02.179954 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa971c0-fe3d-4d85-9e79-0b5e7d4c6a5e None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:02.180242 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa971c0-fe3d-4d85-9e79-0b5e7d4c6a5e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:02.180554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa971c0-fe3d-4d85-9e79-0b5e7d4c6a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:02.201450 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-4aa971c0-fe3d-4d85-9e79-0b5e7d4c6a5e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:02.202651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4aa971c0-fe3d-4d85-9e79-0b5e7d4c6a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:02.211218 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1303/2594] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:02.216337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c942e-f7c3-4457-8670-d9319b5d1fef None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:02.226017 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c942e-f7c3-4457-8670-d9319b5d1fef None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:02.230692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24555fd4-57be-4a06-8f57-22632270587b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:02.231291 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24555fd4-57be-4a06-8f57-22632270587b admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:02.231574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24555fd4-57be-4a06-8f57-22632270587b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:02.231854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24555fd4-57be-4a06-8f57-22632270587b admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:02.232607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24555fd4-57be-4a06-8f57-22632270587b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:02:02.232916 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24555fd4-57be-4a06-8f57-22632270587b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:02.234520 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24555fd4-57be-4a06-8f57-22632270587b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:02.247426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1292/2595] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:02:02.251816 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa971c0-fe3d-4d85-9e79-0b5e7d4c6a5e None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:02.262147 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa971c0-fe3d-4d85-9e79-0b5e7d4c6a5e None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:02.267724 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41645335-1e65-43f0-a63c-73ca8e47294e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:02.268372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41645335-1e65-43f0-a63c-73ca8e47294e admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:02.268659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41645335-1e65-43f0-a63c-73ca8e47294e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:02.268936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41645335-1e65-43f0-a63c-73ca8e47294e admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:02.269614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41645335-1e65-43f0-a63c-73ca8e47294e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:02.271575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41645335-1e65-43f0-a63c-73ca8e47294e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:02.302164 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1304/2596] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:02:02 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:02:02.306259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24555fd4-57be-4a06-8f57-22632270587b admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:02.317964 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24555fd4-57be-4a06-8f57-22632270587b admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:02.382823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa763c93-944d-42be-a113-3f27fa76b200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:02.382823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa763c93-944d-42be-a113-3f27fa76b200 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:02.382823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa763c93-944d-42be-a113-3f27fa76b200 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:02.382823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa763c93-944d-42be-a113-3f27fa76b200 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:02.383412 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa763c93-944d-42be-a113-3f27fa76b200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:02.383412 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa763c93-944d-42be-a113-3f27fa76b200 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:02.390733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:02.406666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1293/2597] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:02:02 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:02:02.410830 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41645335-1e65-43f0-a63c-73ca8e47294e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:02.422822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41645335-1e65-43f0-a63c-73ca8e47294e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:02.427434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c027b4-446c-4e38-9d05-4a839b1b817d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:02.428073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c027b4-446c-4e38-9d05-4a839b1b817d admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:02.428362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c027b4-446c-4e38-9d05-4a839b1b817d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:02.428634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c027b4-446c-4e38-9d05-4a839b1b817d admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:02.429325 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c027b4-446c-4e38-9d05-4a839b1b817d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:02.430948 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c027b4-446c-4e38-9d05-4a839b1b817d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:02.441870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1305/2598] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:02:02.445986 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa763c93-944d-42be-a113-3f27fa76b200 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:02.456714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa763c93-944d-42be-a113-3f27fa76b200 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:02.461481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3160e0-fac4-42a5-929f-7024fb9b786e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:02.462096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3160e0-fac4-42a5-929f-7024fb9b786e admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:02.462381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3160e0-fac4-42a5-929f-7024fb9b786e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:02.462662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3160e0-fac4-42a5-929f-7024fb9b786e admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:02.463388 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3160e0-fac4-42a5-929f-7024fb9b786e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:02.465694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3160e0-fac4-42a5-929f-7024fb9b786e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:02.475990 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1294/2599] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:02:02.480155 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c027b4-446c-4e38-9d05-4a839b1b817d admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:02.491335 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c027b4-446c-4e38-9d05-4a839b1b817d admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:02.496121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccd807-cb07-47ed-a605-6e5ea62eed6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:02.496989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccd807-cb07-47ed-a605-6e5ea62eed6e admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:02.497274 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccd807-cb07-47ed-a605-6e5ea62eed6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:02.497547 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccd807-cb07-47ed-a605-6e5ea62eed6e admin admin] PATH_INFO: `/v3/projects/cb8a6513afc8406b9e160da40573a07b/users/e6c7cc5fdb174e9694981d9d4d9f69ca/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:02.513108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccd807-cb07-47ed-a605-6e5ea62eed6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb8a6513afc8406b9e160da40573a07b, user_id=e6c7cc5fdb174e9694981d9d4d9f69ca, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:02.516083 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccd807-cb07-47ed-a605-6e5ea62eed6e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:02.531631 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1306/2600] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:02:02 2022] PUT /identity/v3/projects/cb8a6513afc8406b9e160da40573a07b/users/e6c7cc5fdb174e9694981d9d4d9f69ca/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:02.542105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3dc73-58ed-4b90-94d2-7d89085ccfd9 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:02.542395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3dc73-58ed-4b90-94d2-7d89085ccfd9 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:02.542704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3dc73-58ed-4b90-94d2-7d89085ccfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:02.574350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-34f3dc73-58ed-4b90-94d2-7d89085ccfd9 None None] MFA Rules not processed for user `e6c7cc5fdb174e9694981d9d4d9f69ca`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:02.576426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-34f3dc73-58ed-4b90-94d2-7d89085ccfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:02.645422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1295/2601] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:02.649941 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccd807-cb07-47ed-a605-6e5ea62eed6e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:02.665195 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:02.720379 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccd807-cb07-47ed-a605-6e5ea62eed6e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:02.726187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e97569-e71c-4401-98ae-a08711454f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:02.726713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e97569-e71c-4401-98ae-a08711454f64 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:02.726998 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e97569-e71c-4401-98ae-a08711454f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:02.727276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e97569-e71c-4401-98ae-a08711454f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:02.727964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e97569-e71c-4401-98ae-a08711454f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:02.729964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e97569-e71c-4401-98ae-a08711454f64 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:02.758690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1307/2602] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:02:02 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 19:02:02.763891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3dc73-58ed-4b90-94d2-7d89085ccfd9 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:02.775124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3dc73-58ed-4b90-94d2-7d89085ccfd9 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:02.832928 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093951d-108d-42a0-8917-5ec12721ad76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:02.833554 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093951d-108d-42a0-8917-5ec12721ad76 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:02.833837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093951d-108d-42a0-8917-5ec12721ad76 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:02.834117 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093951d-108d-42a0-8917-5ec12721ad76 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:02.834790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0093951d-108d-42a0-8917-5ec12721ad76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:02.836761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0093951d-108d-42a0-8917-5ec12721ad76 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:02.858770 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1296/2603] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:02:02 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 19:02:02.862935 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e97569-e71c-4401-98ae-a08711454f64 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:02.873808 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e97569-e71c-4401-98ae-a08711454f64 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:02.878419 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529df596-6e4b-4af6-890e-578c5f2fb487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:02.879288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529df596-6e4b-4af6-890e-578c5f2fb487 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:02.879565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529df596-6e4b-4af6-890e-578c5f2fb487 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:02.879836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529df596-6e4b-4af6-890e-578c5f2fb487 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:02.880593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529df596-6e4b-4af6-890e-578c5f2fb487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:02.882187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529df596-6e4b-4af6-890e-578c5f2fb487 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:02.893396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1308/2604] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:02:02.897551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093951d-108d-42a0-8917-5ec12721ad76 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:02.908084 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093951d-108d-42a0-8917-5ec12721ad76 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:02.912750 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287145cf-354d-4b4d-8c60-d5650c3b59e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:02.913414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287145cf-354d-4b4d-8c60-d5650c3b59e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:02.913695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287145cf-354d-4b4d-8c60-d5650c3b59e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:02.913989 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287145cf-354d-4b4d-8c60-d5650c3b59e9 admin admin] PATH_INFO: `/v3/projects/317da5a458a74f67a2267ec235f57710/users/0f6a4521b0cc4ac7a7331a5f8e0cebba/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:02.929068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287145cf-354d-4b4d-8c60-d5650c3b59e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=317da5a458a74f67a2267ec235f57710, user_id=0f6a4521b0cc4ac7a7331a5f8e0cebba, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:02.931769 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287145cf-354d-4b4d-8c60-d5650c3b59e9 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:02.945244 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1297/2605] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:02:02 2022] PUT /identity/v3/projects/317da5a458a74f67a2267ec235f57710/users/0f6a4521b0cc4ac7a7331a5f8e0cebba/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:02.949869 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529df596-6e4b-4af6-890e-578c5f2fb487 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:02.959968 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529df596-6e4b-4af6-890e-578c5f2fb487 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:03.016532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd329bd0-7a97-46fd-a232-39a1c316a9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:03.017069 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd329bd0-7a97-46fd-a232-39a1c316a9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:03.017352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd329bd0-7a97-46fd-a232-39a1c316a9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:03.017627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd329bd0-7a97-46fd-a232-39a1c316a9c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:03.018312 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd329bd0-7a97-46fd-a232-39a1c316a9c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:03.019915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd329bd0-7a97-46fd-a232-39a1c316a9c7 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:03.030475 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1309/2606] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:02:03.034814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287145cf-354d-4b4d-8c60-d5650c3b59e9 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:03.051043 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287145cf-354d-4b4d-8c60-d5650c3b59e9 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:03.056385 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb903d-8214-4eae-9548-be4dacdb63c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:03.057121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb903d-8214-4eae-9548-be4dacdb63c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:03.057477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb903d-8214-4eae-9548-be4dacdb63c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:03.057800 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb903d-8214-4eae-9548-be4dacdb63c8 admin admin] PATH_INFO: `/v3/projects/317da5a458a74f67a2267ec235f57710/users/0f6a4521b0cc4ac7a7331a5f8e0cebba/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:03.061467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb903d-8214-4eae-9548-be4dacdb63c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=317da5a458a74f67a2267ec235f57710, user_id=0f6a4521b0cc4ac7a7331a5f8e0cebba, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:03.064475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb903d-8214-4eae-9548-be4dacdb63c8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:03.078553 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1298/2607] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:02:03 2022] PUT /identity/v3/projects/317da5a458a74f67a2267ec235f57710/users/0f6a4521b0cc4ac7a7331a5f8e0cebba/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:03.090556 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ae35a-77f5-4ac9-8423-f0c3fecb075b None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:03.090847 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ae35a-77f5-4ac9-8423-f0c3fecb075b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:03.091121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ae35a-77f5-4ac9-8423-f0c3fecb075b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:03.119098 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-101ae35a-77f5-4ac9-8423-f0c3fecb075b None None] MFA Rules not processed for user `0f6a4521b0cc4ac7a7331a5f8e0cebba`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:03.120325 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-101ae35a-77f5-4ac9-8423-f0c3fecb075b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:03.189407 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1310/2608] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:03.201683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb903d-8214-4eae-9548-be4dacdb63c8 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:03.203812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-04cb903d-8214-4eae-9548-be4dacdb63c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:03.228150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:03.265813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb903d-8214-4eae-9548-be4dacdb63c8 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:03.269531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899d872-2e9b-422c-a41c-db928a3064fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:03.270089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899d872-2e9b-422c-a41c-db928a3064fd service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:03.270371 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899d872-2e9b-422c-a41c-db928a3064fd service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:03.270647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899d872-2e9b-422c-a41c-db928a3064fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:03.327546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899d872-2e9b-422c-a41c-db928a3064fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:03.329096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899d872-2e9b-422c-a41c-db928a3064fd service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:03.338126 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1299/2609] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:02:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:02:10.021665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ae35a-77f5-4ac9-8423-f0c3fecb075b None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:10.031282 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ae35a-77f5-4ac9-8423-f0c3fecb075b None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:10.035046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bea7db-0297-4922-8e61-7737bbf2fa7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:10.035667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bea7db-0297-4922-8e61-7737bbf2fa7d service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:10.035947 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bea7db-0297-4922-8e61-7737bbf2fa7d service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:10.036222 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bea7db-0297-4922-8e61-7737bbf2fa7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:10.043396 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:10.046486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bea7db-0297-4922-8e61-7737bbf2fa7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:10.048098 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bea7db-0297-4922-8e61-7737bbf2fa7d service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:10.059156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1311/2610] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:02:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:02:13.334021 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=39 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:20.054359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:39.093462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899d872-2e9b-422c-a41c-db928a3064fd service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:39.156300 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899d872-2e9b-422c-a41c-db928a3064fd service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:39.160774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdd734-dcc3-482b-a270-e9b7c8d6c204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:39.161406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdd734-dcc3-482b-a270-e9b7c8d6c204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:39.161684 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdd734-dcc3-482b-a270-e9b7c8d6c204 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:39.161953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdd734-dcc3-482b-a270-e9b7c8d6c204 admin admin] PATH_INFO: `/v3/users/e6c7cc5fdb174e9694981d9d4d9f69ca` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:39.163102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdd734-dcc3-482b-a270-e9b7c8d6c204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6c7cc5fdb174e9694981d9d4d9f69ca)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:39.165033 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdd734-dcc3-482b-a270-e9b7c8d6c204 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:39.212538 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bfcdd734-dcc3-482b-a270-e9b7c8d6c204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6c7cc5fdb174e9694981d9d4d9f69ca'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:39.220302 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bfcdd734-dcc3-482b-a270-e9b7c8d6c204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6c7cc5fdb174e9694981d9d4d9f69ca'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:39.232658 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bfcdd734-dcc3-482b-a270-e9b7c8d6c204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6c7cc5fdb174e9694981d9d4d9f69ca'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:39.239477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bfcdd734-dcc3-482b-a270-e9b7c8d6c204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6c7cc5fdb174e9694981d9d4d9f69ca'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:39.240120 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bfcdd734-dcc3-482b-a270-e9b7c8d6c204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6c7cc5fdb174e9694981d9d4d9f69ca'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:39.245979 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1300/2611] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:02:39 2022] DELETE /identity/v3/users/e6c7cc5fdb174e9694981d9d4d9f69ca => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:39.250592 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b7a79-ee3d-4234-a90c-b97a1546f673 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:39.251101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b7a79-ee3d-4234-a90c-b97a1546f673 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:39.251217 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b7a79-ee3d-4234-a90c-b97a1546f673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:39.270508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-e96b7a79-ee3d-4234-a90c-b97a1546f673 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:39.271709 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e96b7a79-ee3d-4234-a90c-b97a1546f673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:39.332780 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1312/2612] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:39.343107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdd734-dcc3-482b-a270-e9b7c8d6c204 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:39.344600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-bfcdd734-dcc3-482b-a270-e9b7c8d6c204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:39.404236 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdd734-dcc3-482b-a270-e9b7c8d6c204 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:39.408178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda1e8e-b486-4973-9932-7e8167b57f3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:39.408833 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda1e8e-b486-4973-9932-7e8167b57f3e service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:39.409117 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda1e8e-b486-4973-9932-7e8167b57f3e service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:39.409416 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda1e8e-b486-4973-9932-7e8167b57f3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:39.418529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda1e8e-b486-4973-9932-7e8167b57f3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:39.420795 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda1e8e-b486-4973-9932-7e8167b57f3e service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:39.431154 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1301/2613] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:02:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:02:39.711256 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b7a79-ee3d-4234-a90c-b97a1546f673 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:39.713213 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e96b7a79-ee3d-4234-a90c-b97a1546f673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:39.728560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b7a79-ee3d-4234-a90c-b97a1546f673 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:39.734483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515c235-3569-4510-bffa-0b1c786327d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:39.735280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515c235-3569-4510-bffa-0b1c786327d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:39.735424 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515c235-3569-4510-bffa-0b1c786327d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:39.736063 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515c235-3569-4510-bffa-0b1c786327d8 admin admin] PATH_INFO: `/v3/projects/cb8a6513afc8406b9e160da40573a07b` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:39.737188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515c235-3569-4510-bffa-0b1c786327d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb8a6513afc8406b9e160da40573a07b)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:39.739115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515c235-3569-4510-bffa-0b1c786327d8 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:39.790421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c515c235-3569-4510-bffa-0b1c786327d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb8a6513afc8406b9e160da40573a07b', 'request_id': 'req-c515c235-3569-4510-bffa-0b1c786327d8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:39.798605 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c515c235-3569-4510-bffa-0b1c786327d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb8a6513afc8406b9e160da40573a07b', 'request_id': 'req-c515c235-3569-4510-bffa-0b1c786327d8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:39.805544 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c515c235-3569-4510-bffa-0b1c786327d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb8a6513afc8406b9e160da40573a07b', 'request_id': 'req-c515c235-3569-4510-bffa-0b1c786327d8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:39.817509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c515c235-3569-4510-bffa-0b1c786327d8 admin admin] The token cache is being invalidate because project cb8a6513afc8406b9e160da40573a07b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:02:39.817801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c515c235-3569-4510-bffa-0b1c786327d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:39.818437 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-c515c235-3569-4510-bffa-0b1c786327d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:39.819965 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1313/2614] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:02:39 2022] DELETE /identity/v3/projects/cb8a6513afc8406b9e160da40573a07b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:39.823991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda1e8e-b486-4973-9932-7e8167b57f3e service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:39.834509 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda1e8e-b486-4973-9932-7e8167b57f3e service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:39.852359 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:39.892696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919dc67-d505-473d-8b4e-cf450bdda879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:39.893340 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919dc67-d505-473d-8b4e-cf450bdda879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:39.893625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919dc67-d505-473d-8b4e-cf450bdda879 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:39.893905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919dc67-d505-473d-8b4e-cf450bdda879 admin admin] PATH_INFO: `/v3/users/0f6a4521b0cc4ac7a7331a5f8e0cebba` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:39.895079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919dc67-d505-473d-8b4e-cf450bdda879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f6a4521b0cc4ac7a7331a5f8e0cebba)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:39.897036 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919dc67-d505-473d-8b4e-cf450bdda879 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:39.945809 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c919dc67-d505-473d-8b4e-cf450bdda879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f6a4521b0cc4ac7a7331a5f8e0cebba'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:39.954061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c919dc67-d505-473d-8b4e-cf450bdda879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f6a4521b0cc4ac7a7331a5f8e0cebba'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:39.966051 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c919dc67-d505-473d-8b4e-cf450bdda879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f6a4521b0cc4ac7a7331a5f8e0cebba'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:39.972900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c919dc67-d505-473d-8b4e-cf450bdda879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f6a4521b0cc4ac7a7331a5f8e0cebba'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:39.973542 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-c919dc67-d505-473d-8b4e-cf450bdda879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f6a4521b0cc4ac7a7331a5f8e0cebba'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:39.979507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1302/2615] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:02:39 2022] DELETE /identity/v3/users/0f6a4521b0cc4ac7a7331a5f8e0cebba => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:40.282359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515c235-3569-4510-bffa-0b1c786327d8 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:40.283851 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c515c235-3569-4510-bffa-0b1c786327d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:40.298366 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515c235-3569-4510-bffa-0b1c786327d8 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:40.358169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240223ec-68f2-4800-942a-de875aa231cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:40.358773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240223ec-68f2-4800-942a-de875aa231cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:40.359076 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240223ec-68f2-4800-942a-de875aa231cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:40.359388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240223ec-68f2-4800-942a-de875aa231cc admin admin] PATH_INFO: `/v3/projects/317da5a458a74f67a2267ec235f57710` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:40.360717 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240223ec-68f2-4800-942a-de875aa231cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=317da5a458a74f67a2267ec235f57710)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:40.362632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240223ec-68f2-4800-942a-de875aa231cc admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:40.394287 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:40.412747 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-240223ec-68f2-4800-942a-de875aa231cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '317da5a458a74f67a2267ec235f57710', 'request_id': 'req-240223ec-68f2-4800-942a-de875aa231cc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:40.418080 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-240223ec-68f2-4800-942a-de875aa231cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '317da5a458a74f67a2267ec235f57710', 'request_id': 'req-240223ec-68f2-4800-942a-de875aa231cc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:40.426659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-240223ec-68f2-4800-942a-de875aa231cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '317da5a458a74f67a2267ec235f57710', 'request_id': 'req-240223ec-68f2-4800-942a-de875aa231cc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:40.438691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-240223ec-68f2-4800-942a-de875aa231cc admin admin] The token cache is being invalidate because project 317da5a458a74f67a2267ec235f57710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:02:40.438981 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-240223ec-68f2-4800-942a-de875aa231cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:40.439617 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-240223ec-68f2-4800-942a-de875aa231cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:40.441188 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1314/2616] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:02:40 2022] DELETE /identity/v3/projects/317da5a458a74f67a2267ec235f57710 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:40.445970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae3b30-96bd-411b-a53a-c8592c99c703 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:40.446461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae3b30-96bd-411b-a53a-c8592c99c703 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:40.446577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae3b30-96bd-411b-a53a-c8592c99c703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:40.467459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-49ae3b30-96bd-411b-a53a-c8592c99c703 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:40.467669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-49ae3b30-96bd-411b-a53a-c8592c99c703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:40.529341 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1303/2617] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:40.537555 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352be6a0-6733-4290-9f1c-4565b266c79e None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:40.537847 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352be6a0-6733-4290-9f1c-4565b266c79e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:40.538128 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352be6a0-6733-4290-9f1c-4565b266c79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:40.559271 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-352be6a0-6733-4290-9f1c-4565b266c79e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:40.560791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-352be6a0-6733-4290-9f1c-4565b266c79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:40.568804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1315/2618] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:40.572955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae3b30-96bd-411b-a53a-c8592c99c703 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:40.582869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae3b30-96bd-411b-a53a-c8592c99c703 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:40.587454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784f3aa-7b35-401a-8575-ee095a682c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:40.588063 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784f3aa-7b35-401a-8575-ee095a682c85 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:40.588379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784f3aa-7b35-401a-8575-ee095a682c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:40.588656 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784f3aa-7b35-401a-8575-ee095a682c85 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:40.589346 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784f3aa-7b35-401a-8575-ee095a682c85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:02:40.589650 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784f3aa-7b35-401a-8575-ee095a682c85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:40.591769 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784f3aa-7b35-401a-8575-ee095a682c85 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:40.603769 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1304/2619] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:02:40.608793 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352be6a0-6733-4290-9f1c-4565b266c79e None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:40.618680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352be6a0-6733-4290-9f1c-4565b266c79e None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:40.623389 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94b392-bede-484a-914a-9fcfff423964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:40.624000 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94b392-bede-484a-914a-9fcfff423964 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:40.624276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94b392-bede-484a-914a-9fcfff423964 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:40.624644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94b392-bede-484a-914a-9fcfff423964 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:40.625339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94b392-bede-484a-914a-9fcfff423964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:40.627235 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94b392-bede-484a-914a-9fcfff423964 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:40.655243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1316/2620] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:02:40 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:02:40.659530 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784f3aa-7b35-401a-8575-ee095a682c85 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:40.670698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784f3aa-7b35-401a-8575-ee095a682c85 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:40.696881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:40.733301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd37647-7444-42bd-83b6-03043ea86063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:40.734112 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd37647-7444-42bd-83b6-03043ea86063 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:40.734518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd37647-7444-42bd-83b6-03043ea86063 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:40.734926 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd37647-7444-42bd-83b6-03043ea86063 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:40.735693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd37647-7444-42bd-83b6-03043ea86063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:40.738612 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd37647-7444-42bd-83b6-03043ea86063 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:40.764326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1305/2621] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:02:40 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:02:40.768633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94b392-bede-484a-914a-9fcfff423964 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:40.780833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94b392-bede-484a-914a-9fcfff423964 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:40.786309 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f72b1-a06b-4090-881b-f9bed0e09726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:40.787018 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f72b1-a06b-4090-881b-f9bed0e09726 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:40.787338 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f72b1-a06b-4090-881b-f9bed0e09726 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:40.787716 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f72b1-a06b-4090-881b-f9bed0e09726 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:40.788535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f72b1-a06b-4090-881b-f9bed0e09726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:40.790368 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f72b1-a06b-4090-881b-f9bed0e09726 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:40.802059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1317/2622] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:02:40.806085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd37647-7444-42bd-83b6-03043ea86063 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:40.816158 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd37647-7444-42bd-83b6-03043ea86063 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:40.820812 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f2c2fc-abf9-4818-9d36-1fd0aba82cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:40.821421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2c2fc-abf9-4818-9d36-1fd0aba82cbb admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:40.821702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2c2fc-abf9-4818-9d36-1fd0aba82cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:40.821977 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2c2fc-abf9-4818-9d36-1fd0aba82cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:40.822668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f2c2fc-abf9-4818-9d36-1fd0aba82cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:40.824257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f2c2fc-abf9-4818-9d36-1fd0aba82cbb admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:40.834582 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1306/2623] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:02:40.838829 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f72b1-a06b-4090-881b-f9bed0e09726 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:40.849476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f72b1-a06b-4090-881b-f9bed0e09726 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:40.854039 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28e610-9e79-4724-8c37-bc12488bb45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:40.854703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28e610-9e79-4724-8c37-bc12488bb45f admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:40.854984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28e610-9e79-4724-8c37-bc12488bb45f admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:40.855261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28e610-9e79-4724-8c37-bc12488bb45f admin admin] PATH_INFO: `/v3/projects/91ca78a6a9c64ce9afcb5b9b3b29cebd/users/14df865c42d141cab5e555698089ddd9/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:40.870781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28e610-9e79-4724-8c37-bc12488bb45f admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ca78a6a9c64ce9afcb5b9b3b29cebd, user_id=14df865c42d141cab5e555698089ddd9, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:40.873611 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28e610-9e79-4724-8c37-bc12488bb45f admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:40.886886 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1318/2624] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:02:40 2022] PUT /identity/v3/projects/91ca78a6a9c64ce9afcb5b9b3b29cebd/users/14df865c42d141cab5e555698089ddd9/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:40.900028 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e4ff6-cb87-44d0-bbb0-8ced2ec242b5 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:40.900338 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e4ff6-cb87-44d0-bbb0-8ced2ec242b5 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:40.900618 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e4ff6-cb87-44d0-bbb0-8ced2ec242b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:40.932381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-6a8e4ff6-cb87-44d0-bbb0-8ced2ec242b5 None None] MFA Rules not processed for user `14df865c42d141cab5e555698089ddd9`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:40.933607 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-6a8e4ff6-cb87-44d0-bbb0-8ced2ec242b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:40.997894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1307/2625] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:41.002351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28e610-9e79-4724-8c37-bc12488bb45f admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:41.063902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28e610-9e79-4724-8c37-bc12488bb45f admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:41.068565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee474d-bb16-4c8a-8995-91a2455b65c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:41.069176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee474d-bb16-4c8a-8995-91a2455b65c0 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:41.069457 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee474d-bb16-4c8a-8995-91a2455b65c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:41.069731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee474d-bb16-4c8a-8995-91a2455b65c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:41.070393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee474d-bb16-4c8a-8995-91a2455b65c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:41.072276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee474d-bb16-4c8a-8995-91a2455b65c0 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:41.100435 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1319/2626] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:02:41 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:02:41.104958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e4ff6-cb87-44d0-bbb0-8ced2ec242b5 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:41.116182 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e4ff6-cb87-44d0-bbb0-8ced2ec242b5 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:41.175855 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b8673-391c-406e-bc24-70c83c07b673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:41.176437 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b8673-391c-406e-bc24-70c83c07b673 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:41.176718 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b8673-391c-406e-bc24-70c83c07b673 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:41.176999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b8673-391c-406e-bc24-70c83c07b673 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:41.177667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b8673-391c-406e-bc24-70c83c07b673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:41.179689 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b8673-391c-406e-bc24-70c83c07b673 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:41.202561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1308/2627] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:02:41 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:02:41.206995 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee474d-bb16-4c8a-8995-91a2455b65c0 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:41.217987 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee474d-bb16-4c8a-8995-91a2455b65c0 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:41.222862 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e4a0f-0876-4c11-b431-5cfe93f050b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:41.223476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e4a0f-0876-4c11-b431-5cfe93f050b2 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:41.223757 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e4a0f-0876-4c11-b431-5cfe93f050b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:41.224053 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e4a0f-0876-4c11-b431-5cfe93f050b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:41.224789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e4a0f-0876-4c11-b431-5cfe93f050b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:41.226388 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e4a0f-0876-4c11-b431-5cfe93f050b2 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:41.237080 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1320/2628] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:02:41.241398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b8673-391c-406e-bc24-70c83c07b673 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:41.252344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b8673-391c-406e-bc24-70c83c07b673 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:41.257611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99d93e-e172-4e65-9b76-1755e7f69233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:41.258357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99d93e-e172-4e65-9b76-1755e7f69233 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:41.258671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99d93e-e172-4e65-9b76-1755e7f69233 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:41.258960 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99d93e-e172-4e65-9b76-1755e7f69233 admin admin] PATH_INFO: `/v3/projects/70479a9582374994bc94e933eb8f48d7/users/f3b43428f0ef47d7bac1ec78f75c7620/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:41.275360 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99d93e-e172-4e65-9b76-1755e7f69233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70479a9582374994bc94e933eb8f48d7, user_id=f3b43428f0ef47d7bac1ec78f75c7620, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:41.278426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99d93e-e172-4e65-9b76-1755e7f69233 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:41.292449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1309/2629] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:02:41 2022] PUT /identity/v3/projects/70479a9582374994bc94e933eb8f48d7/users/f3b43428f0ef47d7bac1ec78f75c7620/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:41.296841 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e4a0f-0876-4c11-b431-5cfe93f050b2 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:41.307591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e4a0f-0876-4c11-b431-5cfe93f050b2 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:41.331253 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:41.369477 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da33dec-6396-4a25-9571-3cae51c94690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:41.370096 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da33dec-6396-4a25-9571-3cae51c94690 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:41.370374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da33dec-6396-4a25-9571-3cae51c94690 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:41.370647 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da33dec-6396-4a25-9571-3cae51c94690 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:41.371328 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da33dec-6396-4a25-9571-3cae51c94690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:41.374061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da33dec-6396-4a25-9571-3cae51c94690 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:41.383911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1321/2630] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:02:41.389029 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99d93e-e172-4e65-9b76-1755e7f69233 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:41.394666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:41.401035 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99d93e-e172-4e65-9b76-1755e7f69233 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:41.406603 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bee69-4171-46d6-9ff1-1aaedfaff764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:41.407347 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bee69-4171-46d6-9ff1-1aaedfaff764 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:41.407673 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bee69-4171-46d6-9ff1-1aaedfaff764 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:41.408002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bee69-4171-46d6-9ff1-1aaedfaff764 admin admin] PATH_INFO: `/v3/projects/70479a9582374994bc94e933eb8f48d7/users/f3b43428f0ef47d7bac1ec78f75c7620/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:41.410234 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bee69-4171-46d6-9ff1-1aaedfaff764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70479a9582374994bc94e933eb8f48d7, user_id=f3b43428f0ef47d7bac1ec78f75c7620, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:41.413062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bee69-4171-46d6-9ff1-1aaedfaff764 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:41.424863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1310/2631] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:02:41 2022] PUT /identity/v3/projects/70479a9582374994bc94e933eb8f48d7/users/f3b43428f0ef47d7bac1ec78f75c7620/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 19:02:41.437865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a90664-90fa-47e3-8fd7-dd50480721ab None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:41.438276 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a90664-90fa-47e3-8fd7-dd50480721ab None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:41.438458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a90664-90fa-47e3-8fd7-dd50480721ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:41.466391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-c7a90664-90fa-47e3-8fd7-dd50480721ab None None] MFA Rules not processed for user `f3b43428f0ef47d7bac1ec78f75c7620`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:41.467593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c7a90664-90fa-47e3-8fd7-dd50480721ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:41.536917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1322/2632] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:41.549469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bee69-4171-46d6-9ff1-1aaedfaff764 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:41.551193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-fd8bee69-4171-46d6-9ff1-1aaedfaff764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:41.610849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bee69-4171-46d6-9ff1-1aaedfaff764 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:41.614622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df0982-b74e-4b08-97ae-714dc79a482f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:41.615237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df0982-b74e-4b08-97ae-714dc79a482f service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:41.615515 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df0982-b74e-4b08-97ae-714dc79a482f service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:41.615785 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df0982-b74e-4b08-97ae-714dc79a482f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:41.625488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df0982-b74e-4b08-97ae-714dc79a482f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:41.626910 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df0982-b74e-4b08-97ae-714dc79a482f service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:41.636496 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1311/2633] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:02:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:02:41.953134 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a90664-90fa-47e3-8fd7-dd50480721ab None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:41.962262 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a90664-90fa-47e3-8fd7-dd50480721ab None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:41.966848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385c5bf-ffc6-4344-b84d-bac94a9c65da tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3b43428f0ef47d7bac1ec78f75c7620', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70479a9582374994bc94e933eb8f48d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:41.967466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385c5bf-ffc6-4344-b84d-bac94a9c65da tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:41.967754 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385c5bf-ffc6-4344-b84d-bac94a9c65da tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:41.968036 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385c5bf-ffc6-4344-b84d-bac94a9c65da tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:41.968789 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385c5bf-ffc6-4344-b84d-bac94a9c65da tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:02:41.969101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385c5bf-ffc6-4344-b84d-bac94a9c65da tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:41.970708 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385c5bf-ffc6-4344-b84d-bac94a9c65da tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:41.982754 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1323/2634] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:02:41.986746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df0982-b74e-4b08-97ae-714dc79a482f service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:41.996003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df0982-b74e-4b08-97ae-714dc79a482f service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:42.001732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020cd55-36ed-4f87-8e40-73066e015480 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3b43428f0ef47d7bac1ec78f75c7620', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70479a9582374994bc94e933eb8f48d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:42.002327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020cd55-36ed-4f87-8e40-73066e015480 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:42.002605 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020cd55-36ed-4f87-8e40-73066e015480 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:42.002878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020cd55-36ed-4f87-8e40-73066e015480 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:42.003546 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020cd55-36ed-4f87-8e40-73066e015480 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:42.005737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020cd55-36ed-4f87-8e40-73066e015480 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:42.033601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1312/2635] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:02:41 2022] POST /identity/v3/projects => generated 355 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 19:02:42.037652 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385c5bf-ffc6-4344-b84d-bac94a9c65da tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:42.048196 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385c5bf-ffc6-4344-b84d-bac94a9c65da tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:42.104043 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e3a92-2b7e-4776-a402-992ad97cd892 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3b43428f0ef47d7bac1ec78f75c7620', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70479a9582374994bc94e933eb8f48d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:42.104694 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e3a92-2b7e-4776-a402-992ad97cd892 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:42.104977 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e3a92-2b7e-4776-a402-992ad97cd892 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:42.105248 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e3a92-2b7e-4776-a402-992ad97cd892 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:42.105933 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e3a92-2b7e-4776-a402-992ad97cd892 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:02:42.107290 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e3a92-2b7e-4776-a402-992ad97cd892 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:42.109038 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e3a92-2b7e-4776-a402-992ad97cd892 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:42.120649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1324/2636] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 19:02:42.159169 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020cd55-36ed-4f87-8e40-73066e015480 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:42.171293 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020cd55-36ed-4f87-8e40-73066e015480 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:42.176050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a6dd9-0668-4428-8402-045b04aeef9a tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3b43428f0ef47d7bac1ec78f75c7620', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70479a9582374994bc94e933eb8f48d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:42.176739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a6dd9-0668-4428-8402-045b04aeef9a tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:42.177054 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a6dd9-0668-4428-8402-045b04aeef9a tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:42.177334 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a6dd9-0668-4428-8402-045b04aeef9a tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] PATH_INFO: `/v3/projects/1d1d4288baad41e286307a3ada76e282` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:42.178522 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a6dd9-0668-4428-8402-045b04aeef9a tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1d1d4288baad41e286307a3ada76e282)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:42.180495 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a6dd9-0668-4428-8402-045b04aeef9a tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:42.234410 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-125a6dd9-0668-4428-8402-045b04aeef9a tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d1d4288baad41e286307a3ada76e282', 'request_id': 'req-125a6dd9-0668-4428-8402-045b04aeef9a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:42.240046 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-125a6dd9-0668-4428-8402-045b04aeef9a tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d1d4288baad41e286307a3ada76e282', 'request_id': 'req-125a6dd9-0668-4428-8402-045b04aeef9a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:42.246794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-125a6dd9-0668-4428-8402-045b04aeef9a tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d1d4288baad41e286307a3ada76e282', 'request_id': 'req-125a6dd9-0668-4428-8402-045b04aeef9a'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:42.260899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-125a6dd9-0668-4428-8402-045b04aeef9a tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] The token cache is being invalidate because project 1d1d4288baad41e286307a3ada76e282 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:02:42.261185 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-125a6dd9-0668-4428-8402-045b04aeef9a tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:42.261826 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-125a6dd9-0668-4428-8402-045b04aeef9a tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:42.263577 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1313/2637] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:02:42 2022] DELETE /identity/v3/projects/1d1d4288baad41e286307a3ada76e282 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:42.468354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e3a92-2b7e-4776-a402-992ad97cd892 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:42.470145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-410e3a92-2b7e-4776-a402-992ad97cd892 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:42.502715 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:42.541844 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e3a92-2b7e-4776-a402-992ad97cd892 tempest-VolumeQuotaClassesTest-2077097118 tempest-VolumeQuotaClassesTest-2077097118-project-admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:42.599845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e0b95-ec82-479f-a589-718ff1ed5f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:42.600520 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e0b95-ec82-479f-a589-718ff1ed5f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:42.600799 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e0b95-ec82-479f-a589-718ff1ed5f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:42.601068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e0b95-ec82-479f-a589-718ff1ed5f3e admin admin] PATH_INFO: `/v3/users/14df865c42d141cab5e555698089ddd9` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:42.602227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e0b95-ec82-479f-a589-718ff1ed5f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=14df865c42d141cab5e555698089ddd9)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:42.604147 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e0b95-ec82-479f-a589-718ff1ed5f3e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:42.655251 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-bf7e0b95-ec82-479f-a589-718ff1ed5f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14df865c42d141cab5e555698089ddd9'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:42.662722 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-bf7e0b95-ec82-479f-a589-718ff1ed5f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14df865c42d141cab5e555698089ddd9'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:42.675579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-bf7e0b95-ec82-479f-a589-718ff1ed5f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14df865c42d141cab5e555698089ddd9'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:42.682432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-bf7e0b95-ec82-479f-a589-718ff1ed5f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14df865c42d141cab5e555698089ddd9'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:42.683092 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-bf7e0b95-ec82-479f-a589-718ff1ed5f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14df865c42d141cab5e555698089ddd9'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:42.689319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1325/2638] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:02:42 2022] DELETE /identity/v3/users/14df865c42d141cab5e555698089ddd9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:42.693921 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66fc85-4e94-4cf9-8549-6836aaccf49e None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:42.694416 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66fc85-4e94-4cf9-8549-6836aaccf49e None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:42.694652 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66fc85-4e94-4cf9-8549-6836aaccf49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:42.714633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-dc66fc85-4e94-4cf9-8549-6836aaccf49e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:42.715905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-dc66fc85-4e94-4cf9-8549-6836aaccf49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:42.768044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:42.782307 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1314/2639] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:42.792077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e0b95-ec82-479f-a589-718ff1ed5f3e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:42.793557 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-bf7e0b95-ec82-479f-a589-718ff1ed5f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:42.852504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e0b95-ec82-479f-a589-718ff1ed5f3e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:42.856211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8271e9d-9b18-4b4b-86de-9f7f3d9445f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:42.856899 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8271e9d-9b18-4b4b-86de-9f7f3d9445f7 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:42.857186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8271e9d-9b18-4b4b-86de-9f7f3d9445f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:42.857465 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8271e9d-9b18-4b4b-86de-9f7f3d9445f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:42.864627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:42.866906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8271e9d-9b18-4b4b-86de-9f7f3d9445f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:42.869277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8271e9d-9b18-4b4b-86de-9f7f3d9445f7 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:42.881037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1326/2640] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:02:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:02:43.161057 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66fc85-4e94-4cf9-8549-6836aaccf49e None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:43.162539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-dc66fc85-4e94-4cf9-8549-6836aaccf49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:43.175251 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66fc85-4e94-4cf9-8549-6836aaccf49e None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:43.179929 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2fda2-643c-4f3b-a8a9-8fd97463350f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:43.180617 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2fda2-643c-4f3b-a8a9-8fd97463350f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:43.180902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2fda2-643c-4f3b-a8a9-8fd97463350f admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:43.181176 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2fda2-643c-4f3b-a8a9-8fd97463350f admin admin] PATH_INFO: `/v3/projects/91ca78a6a9c64ce9afcb5b9b3b29cebd` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:43.182363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2fda2-643c-4f3b-a8a9-8fd97463350f admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ca78a6a9c64ce9afcb5b9b3b29cebd)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:43.184982 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2fda2-643c-4f3b-a8a9-8fd97463350f admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:43.234114 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-14c2fda2-643c-4f3b-a8a9-8fd97463350f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ca78a6a9c64ce9afcb5b9b3b29cebd', 'request_id': 'req-14c2fda2-643c-4f3b-a8a9-8fd97463350f'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:43.239541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-14c2fda2-643c-4f3b-a8a9-8fd97463350f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ca78a6a9c64ce9afcb5b9b3b29cebd', 'request_id': 'req-14c2fda2-643c-4f3b-a8a9-8fd97463350f'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:43.245670 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-14c2fda2-643c-4f3b-a8a9-8fd97463350f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ca78a6a9c64ce9afcb5b9b3b29cebd', 'request_id': 'req-14c2fda2-643c-4f3b-a8a9-8fd97463350f'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:43.259150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-14c2fda2-643c-4f3b-a8a9-8fd97463350f admin admin] The token cache is being invalidate because project 91ca78a6a9c64ce9afcb5b9b3b29cebd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:02:43.259454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-14c2fda2-643c-4f3b-a8a9-8fd97463350f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:43.260110 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-14c2fda2-643c-4f3b-a8a9-8fd97463350f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:43.261713 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1315/2641] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:02:43 2022] DELETE /identity/v3/projects/91ca78a6a9c64ce9afcb5b9b3b29cebd => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:43.265752 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8271e9d-9b18-4b4b-86de-9f7f3d9445f7 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:43.275332 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8271e9d-9b18-4b4b-86de-9f7f3d9445f7 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:43.336154 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13d47a-266a-4be2-b422-7c0a3e90e481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:43.336831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13d47a-266a-4be2-b422-7c0a3e90e481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:43.337108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13d47a-266a-4be2-b422-7c0a3e90e481 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:43.337378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13d47a-266a-4be2-b422-7c0a3e90e481 admin admin] PATH_INFO: `/v3/users/f3b43428f0ef47d7bac1ec78f75c7620` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:43.338550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13d47a-266a-4be2-b422-7c0a3e90e481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b43428f0ef47d7bac1ec78f75c7620)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:43.340490 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13d47a-266a-4be2-b422-7c0a3e90e481 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:43.387650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ed13d47a-266a-4be2-b422-7c0a3e90e481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b43428f0ef47d7bac1ec78f75c7620'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:43.395212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ed13d47a-266a-4be2-b422-7c0a3e90e481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b43428f0ef47d7bac1ec78f75c7620'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:43.407608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ed13d47a-266a-4be2-b422-7c0a3e90e481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b43428f0ef47d7bac1ec78f75c7620'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:43.414715 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ed13d47a-266a-4be2-b422-7c0a3e90e481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b43428f0ef47d7bac1ec78f75c7620'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:43.415382 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ed13d47a-266a-4be2-b422-7c0a3e90e481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b43428f0ef47d7bac1ec78f75c7620'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:43.422785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1327/2642] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:02:43 2022] DELETE /identity/v3/users/f3b43428f0ef47d7bac1ec78f75c7620 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:43.713944 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2fda2-643c-4f3b-a8a9-8fd97463350f admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:43.716018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-14c2fda2-643c-4f3b-a8a9-8fd97463350f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:43.729931 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2fda2-643c-4f3b-a8a9-8fd97463350f admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:43.790324 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743959d-62e6-46cc-9a90-10328aec9280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:43.790973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743959d-62e6-46cc-9a90-10328aec9280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:43.791265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743959d-62e6-46cc-9a90-10328aec9280 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:43.791545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743959d-62e6-46cc-9a90-10328aec9280 admin admin] PATH_INFO: `/v3/projects/70479a9582374994bc94e933eb8f48d7` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:43.792760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743959d-62e6-46cc-9a90-10328aec9280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70479a9582374994bc94e933eb8f48d7)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:43.795417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743959d-62e6-46cc-9a90-10328aec9280 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:43.842991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7743959d-62e6-46cc-9a90-10328aec9280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70479a9582374994bc94e933eb8f48d7', 'request_id': 'req-7743959d-62e6-46cc-9a90-10328aec9280'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:43.848397 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7743959d-62e6-46cc-9a90-10328aec9280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70479a9582374994bc94e933eb8f48d7', 'request_id': 'req-7743959d-62e6-46cc-9a90-10328aec9280'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:43.854428 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7743959d-62e6-46cc-9a90-10328aec9280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70479a9582374994bc94e933eb8f48d7', 'request_id': 'req-7743959d-62e6-46cc-9a90-10328aec9280'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:43.868967 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7743959d-62e6-46cc-9a90-10328aec9280 admin admin] The token cache is being invalidate because project 70479a9582374994bc94e933eb8f48d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:02:43.869242 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7743959d-62e6-46cc-9a90-10328aec9280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:43.869865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7743959d-62e6-46cc-9a90-10328aec9280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:43.871509 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1316/2643] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:02:43 2022] DELETE /identity/v3/projects/70479a9582374994bc94e933eb8f48d7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:43.876858 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc4c5f-ca98-4bca-892c-e6131f99cc96 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:43.877372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc4c5f-ca98-4bca-892c-e6131f99cc96 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:43.877488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc4c5f-ca98-4bca-892c-e6131f99cc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:43.899413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-86dc4c5f-ca98-4bca-892c-e6131f99cc96 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:43.900680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-86dc4c5f-ca98-4bca-892c-e6131f99cc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:43.967271 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1328/2644] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:43.976023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fac243-2fe0-497d-8d31-16cb15f21629 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:43.976343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fac243-2fe0-497d-8d31-16cb15f21629 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:43.976623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fac243-2fe0-497d-8d31-16cb15f21629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:43.991829 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:43.998855 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-d4fac243-2fe0-497d-8d31-16cb15f21629 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:44.000912 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d4fac243-2fe0-497d-8d31-16cb15f21629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:44.009118 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1317/2645] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:44.014088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc4c5f-ca98-4bca-892c-e6131f99cc96 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:44.023675 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc4c5f-ca98-4bca-892c-e6131f99cc96 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:44.029214 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9cb8a-caa1-4636-8fae-e4c765424d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:44.029648 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9cb8a-caa1-4636-8fae-e4c765424d09 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:44.029931 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9cb8a-caa1-4636-8fae-e4c765424d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:44.030205 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9cb8a-caa1-4636-8fae-e4c765424d09 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:44.030900 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9cb8a-caa1-4636-8fae-e4c765424d09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:02:44.031202 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9cb8a-caa1-4636-8fae-e4c765424d09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:44.032834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9cb8a-caa1-4636-8fae-e4c765424d09 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:44.044613 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1329/2646] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:02:44.049099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fac243-2fe0-497d-8d31-16cb15f21629 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:44.058361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fac243-2fe0-497d-8d31-16cb15f21629 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:44.063315 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530537f2-afca-4d4f-a0ed-7b1fadf0204b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:44.063916 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530537f2-afca-4d4f-a0ed-7b1fadf0204b admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:44.064195 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530537f2-afca-4d4f-a0ed-7b1fadf0204b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:44.064488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530537f2-afca-4d4f-a0ed-7b1fadf0204b admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:44.065152 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530537f2-afca-4d4f-a0ed-7b1fadf0204b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:44.067033 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530537f2-afca-4d4f-a0ed-7b1fadf0204b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:44.095598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1318/2647] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:02:44 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:02:44.100046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9cb8a-caa1-4636-8fae-e4c765424d09 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:44.105234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:44.111792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9cb8a-caa1-4636-8fae-e4c765424d09 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:44.171033 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22965344-9187-4c1b-84f2-9276b8724794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:44.171762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22965344-9187-4c1b-84f2-9276b8724794 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:44.172103 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22965344-9187-4c1b-84f2-9276b8724794 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:44.172487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22965344-9187-4c1b-84f2-9276b8724794 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:44.173229 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22965344-9187-4c1b-84f2-9276b8724794 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:44.175331 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22965344-9187-4c1b-84f2-9276b8724794 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:44.197796 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1330/2648] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:02:44 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:02:44.202631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530537f2-afca-4d4f-a0ed-7b1fadf0204b admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:44.212917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530537f2-afca-4d4f-a0ed-7b1fadf0204b admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:44.218627 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ae083-df0a-4e7c-bcea-3f909de50cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:44.219255 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ae083-df0a-4e7c-bcea-3f909de50cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:44.219560 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ae083-df0a-4e7c-bcea-3f909de50cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:44.219831 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ae083-df0a-4e7c-bcea-3f909de50cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:44.220557 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ae083-df0a-4e7c-bcea-3f909de50cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:44.222130 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ae083-df0a-4e7c-bcea-3f909de50cd3 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:44.232490 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1319/2649] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:02:44.236310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22965344-9187-4c1b-84f2-9276b8724794 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:44.246280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22965344-9187-4c1b-84f2-9276b8724794 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:44.252160 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89959a51-cb74-4b81-9f0c-8387e6f8cd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:44.252803 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89959a51-cb74-4b81-9f0c-8387e6f8cd68 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:44.253085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89959a51-cb74-4b81-9f0c-8387e6f8cd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:44.253361 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89959a51-cb74-4b81-9f0c-8387e6f8cd68 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:44.254056 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89959a51-cb74-4b81-9f0c-8387e6f8cd68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:44.255912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89959a51-cb74-4b81-9f0c-8387e6f8cd68 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:44.267362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1331/2650] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:02:44.271436 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ae083-df0a-4e7c-bcea-3f909de50cd3 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:44.281803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ae083-df0a-4e7c-bcea-3f909de50cd3 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:44.286518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2657bc-5bce-4e5f-a043-f3e12818b18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:44.287191 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2657bc-5bce-4e5f-a043-f3e12818b18b admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:44.287477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2657bc-5bce-4e5f-a043-f3e12818b18b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:44.287764 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2657bc-5bce-4e5f-a043-f3e12818b18b admin admin] PATH_INFO: `/v3/projects/1bfec1e8594c44f988555277373417e9/users/01267c7ecb044fff89659f11d57df6c0/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:44.303388 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2657bc-5bce-4e5f-a043-f3e12818b18b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bfec1e8594c44f988555277373417e9, user_id=01267c7ecb044fff89659f11d57df6c0, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:44.306140 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2657bc-5bce-4e5f-a043-f3e12818b18b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:44.320525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1320/2651] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:02:44 2022] PUT /identity/v3/projects/1bfec1e8594c44f988555277373417e9/users/01267c7ecb044fff89659f11d57df6c0/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:44.331705 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5461c52-cda3-48f9-a78a-bebc0a6b6f8c None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:44.331991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5461c52-cda3-48f9-a78a-bebc0a6b6f8c None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:44.332266 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5461c52-cda3-48f9-a78a-bebc0a6b6f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:44.360269 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-f5461c52-cda3-48f9-a78a-bebc0a6b6f8c None None] MFA Rules not processed for user `01267c7ecb044fff89659f11d57df6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:44.361432 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f5461c52-cda3-48f9-a78a-bebc0a6b6f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:44.427334 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1332/2652] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:44.432043 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2657bc-5bce-4e5f-a043-f3e12818b18b admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:44.494686 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2657bc-5bce-4e5f-a043-f3e12818b18b admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:44.500108 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8940e76-386c-4b73-810d-54f0a29c73b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:44.500891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8940e76-386c-4b73-810d-54f0a29c73b9 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:44.501175 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8940e76-386c-4b73-810d-54f0a29c73b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:44.501447 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8940e76-386c-4b73-810d-54f0a29c73b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:44.502121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8940e76-386c-4b73-810d-54f0a29c73b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:44.504007 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8940e76-386c-4b73-810d-54f0a29c73b9 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:44.531158 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1321/2653] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:02:44 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:02:44.535288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5461c52-cda3-48f9-a78a-bebc0a6b6f8c None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:44.546051 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5461c52-cda3-48f9-a78a-bebc0a6b6f8c None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:44.601871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd338207-8fbe-4e32-8719-f16d902b368a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:44.602491 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd338207-8fbe-4e32-8719-f16d902b368a admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:44.602771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd338207-8fbe-4e32-8719-f16d902b368a admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:44.603050 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd338207-8fbe-4e32-8719-f16d902b368a admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:44.603737 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd338207-8fbe-4e32-8719-f16d902b368a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:44.605704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd338207-8fbe-4e32-8719-f16d902b368a admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:44.623420 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:44.633647 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1333/2654] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:02:44 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 19:02:44.634521 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8940e76-386c-4b73-810d-54f0a29c73b9 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:44.645182 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8940e76-386c-4b73-810d-54f0a29c73b9 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:44.650571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4273cb-fb13-4878-b47f-41dd20e79378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:44.651187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4273cb-fb13-4878-b47f-41dd20e79378 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:44.651464 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4273cb-fb13-4878-b47f-41dd20e79378 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:44.651740 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4273cb-fb13-4878-b47f-41dd20e79378 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:44.652506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4273cb-fb13-4878-b47f-41dd20e79378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:44.654649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4273cb-fb13-4878-b47f-41dd20e79378 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:44.665390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1322/2655] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 19:02:44.670243 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd338207-8fbe-4e32-8719-f16d902b368a admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:44.681892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd338207-8fbe-4e32-8719-f16d902b368a admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:44.688746 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f035434-a83a-4b48-b225-e099bb433ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:44.689483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f035434-a83a-4b48-b225-e099bb433ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:44.689794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f035434-a83a-4b48-b225-e099bb433ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:44.690097 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f035434-a83a-4b48-b225-e099bb433ac8 admin admin] PATH_INFO: `/v3/projects/b0bb8fa16ebd4ed09bf86ae744ba0b08/users/002e9b92448e4c1bb424b0167c1ffcd7/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:44.707181 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f035434-a83a-4b48-b225-e099bb433ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0bb8fa16ebd4ed09bf86ae744ba0b08, user_id=002e9b92448e4c1bb424b0167c1ffcd7, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:44.710272 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f035434-a83a-4b48-b225-e099bb433ac8 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:44.724877 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1334/2656] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:02:44 2022] PUT /identity/v3/projects/b0bb8fa16ebd4ed09bf86ae744ba0b08/users/002e9b92448e4c1bb424b0167c1ffcd7/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:44.729073 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4273cb-fb13-4878-b47f-41dd20e79378 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:44.740159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4273cb-fb13-4878-b47f-41dd20e79378 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:44.798542 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3da65-7c53-4567-adb5-910de028c7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:44.799196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3da65-7c53-4567-adb5-910de028c7c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:44.799504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3da65-7c53-4567-adb5-910de028c7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:44.799783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3da65-7c53-4567-adb5-910de028c7c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:44.800517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3da65-7c53-4567-adb5-910de028c7c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:44.802343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3da65-7c53-4567-adb5-910de028c7c0 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:44.814043 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1323/2657] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:02:44.818273 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f035434-a83a-4b48-b225-e099bb433ac8 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:44.829570 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f035434-a83a-4b48-b225-e099bb433ac8 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:44.834223 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc74a07-e6c3-4509-806a-f07aecc4a7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:44.834891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc74a07-e6c3-4509-806a-f07aecc4a7b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:44.835169 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc74a07-e6c3-4509-806a-f07aecc4a7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:44.835439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc74a07-e6c3-4509-806a-f07aecc4a7b8 admin admin] PATH_INFO: `/v3/projects/b0bb8fa16ebd4ed09bf86ae744ba0b08/users/002e9b92448e4c1bb424b0167c1ffcd7/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:44.837402 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc74a07-e6c3-4509-806a-f07aecc4a7b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0bb8fa16ebd4ed09bf86ae744ba0b08, user_id=002e9b92448e4c1bb424b0167c1ffcd7, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:44.840037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc74a07-e6c3-4509-806a-f07aecc4a7b8 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:44.850809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1335/2658] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:02:44 2022] PUT /identity/v3/projects/b0bb8fa16ebd4ed09bf86ae744ba0b08/users/002e9b92448e4c1bb424b0167c1ffcd7/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:44.861986 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf7f3f-93c8-4fca-b21a-1392d1296725 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:44.862334 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf7f3f-93c8-4fca-b21a-1392d1296725 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:44.862618 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf7f3f-93c8-4fca-b21a-1392d1296725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:44.888869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:44.892296 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-d9cf7f3f-93c8-4fca-b21a-1392d1296725 None None] MFA Rules not processed for user `002e9b92448e4c1bb424b0167c1ffcd7`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:44.894096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d9cf7f3f-93c8-4fca-b21a-1392d1296725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:44.964671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1324/2659] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:44 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:44.975346 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc74a07-e6c3-4509-806a-f07aecc4a7b8 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:44.977107 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8cc74a07-e6c3-4509-806a-f07aecc4a7b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:45.036822 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc74a07-e6c3-4509-806a-f07aecc4a7b8 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:45.040474 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f2a9d-d675-47f2-afeb-6df731e38b55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:45.041084 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f2a9d-d675-47f2-afeb-6df731e38b55 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:45.041360 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f2a9d-d675-47f2-afeb-6df731e38b55 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:45.041629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f2a9d-d675-47f2-afeb-6df731e38b55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:45.051073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f2a9d-d675-47f2-afeb-6df731e38b55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:45.052528 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f2a9d-d675-47f2-afeb-6df731e38b55 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:45.061883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1336/2660] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:02:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:02:45.443139 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf7f3f-93c8-4fca-b21a-1392d1296725 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:45.507441 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf7f3f-93c8-4fca-b21a-1392d1296725 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:45.512086 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89308212-236b-46a5-97d3-023db1380281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:45.513927 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89308212-236b-46a5-97d3-023db1380281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:45.514239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89308212-236b-46a5-97d3-023db1380281 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:45.514518 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89308212-236b-46a5-97d3-023db1380281 admin admin] PATH_INFO: `/v3/users/01267c7ecb044fff89659f11d57df6c0` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:45.515681 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89308212-236b-46a5-97d3-023db1380281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01267c7ecb044fff89659f11d57df6c0)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:45.517610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89308212-236b-46a5-97d3-023db1380281 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:45.564556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-89308212-236b-46a5-97d3-023db1380281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01267c7ecb044fff89659f11d57df6c0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:45.572508 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-89308212-236b-46a5-97d3-023db1380281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01267c7ecb044fff89659f11d57df6c0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:45.585131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-89308212-236b-46a5-97d3-023db1380281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01267c7ecb044fff89659f11d57df6c0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:45.592871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-89308212-236b-46a5-97d3-023db1380281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01267c7ecb044fff89659f11d57df6c0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:45.593517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-89308212-236b-46a5-97d3-023db1380281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01267c7ecb044fff89659f11d57df6c0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:45.599625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1325/2661] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:02:45 2022] DELETE /identity/v3/users/01267c7ecb044fff89659f11d57df6c0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:45.604413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce549cb-6aba-44da-94b4-74e8d25832ed None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:45.604703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce549cb-6aba-44da-94b4-74e8d25832ed None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:45.604980 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce549cb-6aba-44da-94b4-74e8d25832ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:45.623522 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-8ce549cb-6aba-44da-94b4-74e8d25832ed None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:45.624761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8ce549cb-6aba-44da-94b4-74e8d25832ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:45.684639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1337/2662] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:45.694253 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89308212-236b-46a5-97d3-023db1380281 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:45.695636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-89308212-236b-46a5-97d3-023db1380281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:45.717668 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:45.758041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89308212-236b-46a5-97d3-023db1380281 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:45.761746 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da19c6-91b8-499a-a55a-6b7dfac6f0db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:45.762525 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da19c6-91b8-499a-a55a-6b7dfac6f0db service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:45.763806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da19c6-91b8-499a-a55a-6b7dfac6f0db service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:45.764086 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da19c6-91b8-499a-a55a-6b7dfac6f0db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:45.773196 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da19c6-91b8-499a-a55a-6b7dfac6f0db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:45.774617 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da19c6-91b8-499a-a55a-6b7dfac6f0db service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:45.784675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1326/2663] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:02:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:02:46.069215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce549cb-6aba-44da-94b4-74e8d25832ed None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:46.070665 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8ce549cb-6aba-44da-94b4-74e8d25832ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:46.085627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce549cb-6aba-44da-94b4-74e8d25832ed None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:46.090768 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18545b0d-293f-499e-bd47-cffb900aef81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:46.091430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18545b0d-293f-499e-bd47-cffb900aef81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:46.091741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18545b0d-293f-499e-bd47-cffb900aef81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:46.091949 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18545b0d-293f-499e-bd47-cffb900aef81 admin admin] PATH_INFO: `/v3/projects/1bfec1e8594c44f988555277373417e9` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:46.093228 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18545b0d-293f-499e-bd47-cffb900aef81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bfec1e8594c44f988555277373417e9)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:46.095365 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18545b0d-293f-499e-bd47-cffb900aef81 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:46.145944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-18545b0d-293f-499e-bd47-cffb900aef81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bfec1e8594c44f988555277373417e9', 'request_id': 'req-18545b0d-293f-499e-bd47-cffb900aef81'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:46.151634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-18545b0d-293f-499e-bd47-cffb900aef81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bfec1e8594c44f988555277373417e9', 'request_id': 'req-18545b0d-293f-499e-bd47-cffb900aef81'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:46.154372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:46.159934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-18545b0d-293f-499e-bd47-cffb900aef81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bfec1e8594c44f988555277373417e9', 'request_id': 'req-18545b0d-293f-499e-bd47-cffb900aef81'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:46.173645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-18545b0d-293f-499e-bd47-cffb900aef81 admin admin] The token cache is being invalidate because project 1bfec1e8594c44f988555277373417e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:02:46.173934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-18545b0d-293f-499e-bd47-cffb900aef81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:46.174564 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-18545b0d-293f-499e-bd47-cffb900aef81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:46.176077 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1338/2664] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:02:46 2022] DELETE /identity/v3/projects/1bfec1e8594c44f988555277373417e9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:46.180109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da19c6-91b8-499a-a55a-6b7dfac6f0db service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:46.190393 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da19c6-91b8-499a-a55a-6b7dfac6f0db service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:46.249917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffef2d9-6c5a-458d-8db3-7b098d6cccec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:46.250680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffef2d9-6c5a-458d-8db3-7b098d6cccec admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:46.250968 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffef2d9-6c5a-458d-8db3-7b098d6cccec admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:46.251245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffef2d9-6c5a-458d-8db3-7b098d6cccec admin admin] PATH_INFO: `/v3/users/002e9b92448e4c1bb424b0167c1ffcd7` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:46.252465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffef2d9-6c5a-458d-8db3-7b098d6cccec admin admin] RBAC: Authorizing `identity:delete_user(user_id=002e9b92448e4c1bb424b0167c1ffcd7)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:46.254378 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffef2d9-6c5a-458d-8db3-7b098d6cccec admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:46.302693 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ffef2d9-6c5a-458d-8db3-7b098d6cccec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '002e9b92448e4c1bb424b0167c1ffcd7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:46.310922 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ffef2d9-6c5a-458d-8db3-7b098d6cccec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '002e9b92448e4c1bb424b0167c1ffcd7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:46.325260 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ffef2d9-6c5a-458d-8db3-7b098d6cccec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '002e9b92448e4c1bb424b0167c1ffcd7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:46.333905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ffef2d9-6c5a-458d-8db3-7b098d6cccec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '002e9b92448e4c1bb424b0167c1ffcd7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:46.334639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ffef2d9-6c5a-458d-8db3-7b098d6cccec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '002e9b92448e4c1bb424b0167c1ffcd7'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:46.340686 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1327/2665] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:02:46 2022] DELETE /identity/v3/users/002e9b92448e4c1bb424b0167c1ffcd7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:46.625288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18545b0d-293f-499e-bd47-cffb900aef81 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:46.626742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-18545b0d-293f-499e-bd47-cffb900aef81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:46.643451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18545b0d-293f-499e-bd47-cffb900aef81 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:46.702515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62092286-51b9-4964-b876-31699cbc4b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:46.703186 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62092286-51b9-4964-b876-31699cbc4b32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:46.703451 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62092286-51b9-4964-b876-31699cbc4b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:46.703728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62092286-51b9-4964-b876-31699cbc4b32 admin admin] PATH_INFO: `/v3/projects/b0bb8fa16ebd4ed09bf86ae744ba0b08` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:46.704913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62092286-51b9-4964-b876-31699cbc4b32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0bb8fa16ebd4ed09bf86ae744ba0b08)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:46.706830 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62092286-51b9-4964-b876-31699cbc4b32 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:46.753865 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-62092286-51b9-4964-b876-31699cbc4b32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0bb8fa16ebd4ed09bf86ae744ba0b08', 'request_id': 'req-62092286-51b9-4964-b876-31699cbc4b32'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:46.759600 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-62092286-51b9-4964-b876-31699cbc4b32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0bb8fa16ebd4ed09bf86ae744ba0b08', 'request_id': 'req-62092286-51b9-4964-b876-31699cbc4b32'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:46.766766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-62092286-51b9-4964-b876-31699cbc4b32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0bb8fa16ebd4ed09bf86ae744ba0b08', 'request_id': 'req-62092286-51b9-4964-b876-31699cbc4b32'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:46.777835 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-62092286-51b9-4964-b876-31699cbc4b32 admin admin] The token cache is being invalidate because project b0bb8fa16ebd4ed09bf86ae744ba0b08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:02:46.778787 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-62092286-51b9-4964-b876-31699cbc4b32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:46.779600 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-62092286-51b9-4964-b876-31699cbc4b32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:02:46.781146 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1339/2666] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:02:46 2022] DELETE /identity/v3/projects/b0bb8fa16ebd4ed09bf86ae744ba0b08 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:46.785982 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00c4b4-f78c-47e3-8453-03b76c406a54 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:46.786461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00c4b4-f78c-47e3-8453-03b76c406a54 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:46.786583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00c4b4-f78c-47e3-8453-03b76c406a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:46.806864 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ea00c4b4-f78c-47e3-8453-03b76c406a54 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:46.808100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ea00c4b4-f78c-47e3-8453-03b76c406a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:46.842244 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:46.872845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1328/2667] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:46.880804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9d2e5-8817-4eba-9db0-18a14b712db5 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:46.881090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9d2e5-8817-4eba-9db0-18a14b712db5 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:46.881366 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9d2e5-8817-4eba-9db0-18a14b712db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:46.901337 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-49e9d2e5-8817-4eba-9db0-18a14b712db5 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:46.902597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-49e9d2e5-8817-4eba-9db0-18a14b712db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:46.911227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1340/2668] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:46.915633 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00c4b4-f78c-47e3-8453-03b76c406a54 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:46.925749 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00c4b4-f78c-47e3-8453-03b76c406a54 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:46.930435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8f331-a8d8-4540-a738-62146d3f579c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:46.931053 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8f331-a8d8-4540-a738-62146d3f579c admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:46.931334 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8f331-a8d8-4540-a738-62146d3f579c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:46.931606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8f331-a8d8-4540-a738-62146d3f579c admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:46.932321 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8f331-a8d8-4540-a738-62146d3f579c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:02:46.932635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8f331-a8d8-4540-a738-62146d3f579c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:46.934257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8f331-a8d8-4540-a738-62146d3f579c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:46.946194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1329/2669] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:02:46.951049 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9d2e5-8817-4eba-9db0-18a14b712db5 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:46.961909 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9d2e5-8817-4eba-9db0-18a14b712db5 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:46.967650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8745c5-0374-44ec-a1ca-6c89d2b7af97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:46.968246 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8745c5-0374-44ec-a1ca-6c89d2b7af97 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:46.968561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8745c5-0374-44ec-a1ca-6c89d2b7af97 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:46.968837 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8745c5-0374-44ec-a1ca-6c89d2b7af97 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:46.969517 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8745c5-0374-44ec-a1ca-6c89d2b7af97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:46.971416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8745c5-0374-44ec-a1ca-6c89d2b7af97 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:46.998199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1341/2670] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:02:46 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:02:47.002600 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8f331-a8d8-4540-a738-62146d3f579c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:47.013673 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8f331-a8d8-4540-a738-62146d3f579c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:47.069533 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a48367-8787-4a4c-ab22-df22e19768e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:47.070150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a48367-8787-4a4c-ab22-df22e19768e8 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:47.070437 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a48367-8787-4a4c-ab22-df22e19768e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:47.070717 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a48367-8787-4a4c-ab22-df22e19768e8 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:47.071395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a48367-8787-4a4c-ab22-df22e19768e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:47.073357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a48367-8787-4a4c-ab22-df22e19768e8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:47.096880 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1330/2671] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:02:47 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 19:02:47.101126 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8745c5-0374-44ec-a1ca-6c89d2b7af97 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:47.112130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8745c5-0374-44ec-a1ca-6c89d2b7af97 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:47.117157 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a5f21-554c-49c9-af1f-a7d4ff8d78dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:47.117824 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a5f21-554c-49c9-af1f-a7d4ff8d78dd admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:47.118140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a5f21-554c-49c9-af1f-a7d4ff8d78dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:47.118460 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a5f21-554c-49c9-af1f-a7d4ff8d78dd admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:47.119202 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a5f21-554c-49c9-af1f-a7d4ff8d78dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:47.120988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a5f21-554c-49c9-af1f-a7d4ff8d78dd admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:47.131639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1342/2672] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:02:47.135505 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a48367-8787-4a4c-ab22-df22e19768e8 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:47.146432 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a48367-8787-4a4c-ab22-df22e19768e8 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:47.151905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe80574-8752-4b2a-91cf-94d988127f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:47.152573 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe80574-8752-4b2a-91cf-94d988127f43 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:47.152860 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe80574-8752-4b2a-91cf-94d988127f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:47.153758 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe80574-8752-4b2a-91cf-94d988127f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:47.154473 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe80574-8752-4b2a-91cf-94d988127f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:47.156072 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe80574-8752-4b2a-91cf-94d988127f43 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:47.167404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1331/2673] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:02:47.172048 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a5f21-554c-49c9-af1f-a7d4ff8d78dd admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:47.182440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a5f21-554c-49c9-af1f-a7d4ff8d78dd admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:47.187559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bba345-748c-4bbb-8cda-5b1cec1d9278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:47.188271 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bba345-748c-4bbb-8cda-5b1cec1d9278 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:47.188614 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bba345-748c-4bbb-8cda-5b1cec1d9278 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:47.188915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bba345-748c-4bbb-8cda-5b1cec1d9278 admin admin] PATH_INFO: `/v3/projects/7c6c0acf801848c9bdbc9bdc64d24866/users/7f2b3d80a68a4a34ae19056f4d63ce8b/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:47.204957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bba345-748c-4bbb-8cda-5b1cec1d9278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c6c0acf801848c9bdbc9bdc64d24866, user_id=7f2b3d80a68a4a34ae19056f4d63ce8b, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:47.207638 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bba345-748c-4bbb-8cda-5b1cec1d9278 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:47.221851 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1343/2674] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:02:47 2022] PUT /identity/v3/projects/7c6c0acf801848c9bdbc9bdc64d24866/users/7f2b3d80a68a4a34ae19056f4d63ce8b/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:47.226955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e70dd4-699f-4449-83f8-423d3d180148 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:47.227250 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e70dd4-699f-4449-83f8-423d3d180148 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:47.227528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e70dd4-699f-4449-83f8-423d3d180148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:47.247574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-71e70dd4-699f-4449-83f8-423d3d180148 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:47.248838 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-71e70dd4-699f-4449-83f8-423d3d180148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:47.310667 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1332/2675] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:47.320062 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bba345-748c-4bbb-8cda-5b1cec1d9278 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:47.321824 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-24bba345-748c-4bbb-8cda-5b1cec1d9278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:47.343548 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:47.386783 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bba345-748c-4bbb-8cda-5b1cec1d9278 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:47.390844 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff6bc7-5d1c-40d6-a117-4be4d651550f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:47.391466 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff6bc7-5d1c-40d6-a117-4be4d651550f service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:47.391752 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff6bc7-5d1c-40d6-a117-4be4d651550f service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:47.392027 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff6bc7-5d1c-40d6-a117-4be4d651550f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:47.401183 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff6bc7-5d1c-40d6-a117-4be4d651550f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:47.402623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff6bc7-5d1c-40d6-a117-4be4d651550f service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:47.412764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1344/2676] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:02:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:02:48.038045 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3856077-eb44-4ef0-b8be-73b73ef96625 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:48.038598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3856077-eb44-4ef0-b8be-73b73ef96625 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:48.038857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3856077-eb44-4ef0-b8be-73b73ef96625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:48.070786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-c3856077-eb44-4ef0-b8be-73b73ef96625 None None] MFA Rules not processed for user `7f2b3d80a68a4a34ae19056f4d63ce8b`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:48.072033 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c3856077-eb44-4ef0-b8be-73b73ef96625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:48.110817 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:48.147706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1333/2677] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:48.152481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff6bc7-5d1c-40d6-a117-4be4d651550f service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:48.162715 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff6bc7-5d1c-40d6-a117-4be4d651550f service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:48.167733 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8350e50-ee56-4d47-aeb2-a07defdfdfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:48.168397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8350e50-ee56-4d47-aeb2-a07defdfdfa1 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:48.168912 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8350e50-ee56-4d47-aeb2-a07defdfdfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:48.169146 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8350e50-ee56-4d47-aeb2-a07defdfdfa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:48.170073 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8350e50-ee56-4d47-aeb2-a07defdfdfa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:48.172009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8350e50-ee56-4d47-aeb2-a07defdfdfa1 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:48.202112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1345/2678] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:02:48 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:02:48.206483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3856077-eb44-4ef0-b8be-73b73ef96625 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:48.217892 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3856077-eb44-4ef0-b8be-73b73ef96625 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:48.277594 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52faf7e7-bbb7-4e46-9f4e-93ae8d935adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:48.278272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faf7e7-bbb7-4e46-9f4e-93ae8d935adf admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:48.278559 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faf7e7-bbb7-4e46-9f4e-93ae8d935adf admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:48.278836 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faf7e7-bbb7-4e46-9f4e-93ae8d935adf admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:48.279986 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52faf7e7-bbb7-4e46-9f4e-93ae8d935adf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:48.283386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52faf7e7-bbb7-4e46-9f4e-93ae8d935adf admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:48.313324 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1334/2679] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:02:48 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:02:48.319330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8350e50-ee56-4d47-aeb2-a07defdfdfa1 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:48.334544 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8350e50-ee56-4d47-aeb2-a07defdfdfa1 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:48.339778 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd925926-4d40-452c-bcbe-94906b8bd8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:48.340612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd925926-4d40-452c-bcbe-94906b8bd8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:48.341083 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd925926-4d40-452c-bcbe-94906b8bd8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:48.342226 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd925926-4d40-452c-bcbe-94906b8bd8e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:48.343326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd925926-4d40-452c-bcbe-94906b8bd8e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:48.345006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd925926-4d40-452c-bcbe-94906b8bd8e2 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:48.357844 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1346/2680] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:02:48.362585 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52faf7e7-bbb7-4e46-9f4e-93ae8d935adf admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:48.374805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52faf7e7-bbb7-4e46-9f4e-93ae8d935adf admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:48.379642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c5d73-2b78-45b2-acbb-97ad1302846d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:48.380630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c5d73-2b78-45b2-acbb-97ad1302846d admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:48.380909 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c5d73-2b78-45b2-acbb-97ad1302846d admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:48.381184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c5d73-2b78-45b2-acbb-97ad1302846d admin admin] PATH_INFO: `/v3/projects/81ef73036f0d44c8bc1f5c1ac8d3abfd/users/627207fc754d44d9afb7b3f0452d1ca0/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:48.398574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c5d73-2b78-45b2-acbb-97ad1302846d admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ef73036f0d44c8bc1f5c1ac8d3abfd, user_id=627207fc754d44d9afb7b3f0452d1ca0, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:48.401284 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c5d73-2b78-45b2-acbb-97ad1302846d admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:48.416630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1335/2681] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:02:48 2022] PUT /identity/v3/projects/81ef73036f0d44c8bc1f5c1ac8d3abfd/users/627207fc754d44d9afb7b3f0452d1ca0/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:48.422501 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd925926-4d40-452c-bcbe-94906b8bd8e2 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:48.435915 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd925926-4d40-452c-bcbe-94906b8bd8e2 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:48.500983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713185f-2590-41eb-b1a6-2ca2fbb28906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:48.501518 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713185f-2590-41eb-b1a6-2ca2fbb28906 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:48.501794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713185f-2590-41eb-b1a6-2ca2fbb28906 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:48.502063 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713185f-2590-41eb-b1a6-2ca2fbb28906 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:48.502749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713185f-2590-41eb-b1a6-2ca2fbb28906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:48.504359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713185f-2590-41eb-b1a6-2ca2fbb28906 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:48.515008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1347/2682] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:02:48.519295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c5d73-2b78-45b2-acbb-97ad1302846d admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:48.529702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c5d73-2b78-45b2-acbb-97ad1302846d admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:48.534496 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77166613-5ab7-40c6-969c-85a30a6d6421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:48.535165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77166613-5ab7-40c6-969c-85a30a6d6421 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:48.535448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77166613-5ab7-40c6-969c-85a30a6d6421 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:48.535724 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77166613-5ab7-40c6-969c-85a30a6d6421 admin admin] PATH_INFO: `/v3/projects/81ef73036f0d44c8bc1f5c1ac8d3abfd/users/627207fc754d44d9afb7b3f0452d1ca0/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:48.537735 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77166613-5ab7-40c6-969c-85a30a6d6421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ef73036f0d44c8bc1f5c1ac8d3abfd, user_id=627207fc754d44d9afb7b3f0452d1ca0, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:48.540415 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77166613-5ab7-40c6-969c-85a30a6d6421 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:48.551563 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1336/2683] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:02:48 2022] PUT /identity/v3/projects/81ef73036f0d44c8bc1f5c1ac8d3abfd/users/627207fc754d44d9afb7b3f0452d1ca0/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:02:49.436882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e97a51-4586-4ada-b519-bb61fc734e95 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:49.437449 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e97a51-4586-4ada-b519-bb61fc734e95 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:49.437663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e97a51-4586-4ada-b519-bb61fc734e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:49.473234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-50e97a51-4586-4ada-b519-bb61fc734e95 None None] MFA Rules not processed for user `627207fc754d44d9afb7b3f0452d1ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:02:49.474458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-50e97a51-4586-4ada-b519-bb61fc734e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:49.560074 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1348/2684] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:02:49 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:02:49.571318 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77166613-5ab7-40c6-969c-85a30a6d6421 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:49.573101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-77166613-5ab7-40c6-969c-85a30a6d6421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:49.635087 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77166613-5ab7-40c6-969c-85a30a6d6421 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:49.639504 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41185da0-8a97-4ada-940a-8dc9b146ec00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:49.640128 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41185da0-8a97-4ada-940a-8dc9b146ec00 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:49.640561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41185da0-8a97-4ada-940a-8dc9b146ec00 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:49.640862 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41185da0-8a97-4ada-940a-8dc9b146ec00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:49.682455 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:49.698282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41185da0-8a97-4ada-940a-8dc9b146ec00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:49.699936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41185da0-8a97-4ada-940a-8dc9b146ec00 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:49.709634 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1337/2685] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:02:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:02:51.077626 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e97a51-4586-4ada-b519-bb61fc734e95 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:02:51.079202 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-50e97a51-4586-4ada-b519-bb61fc734e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:02:51.092123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e97a51-4586-4ada-b519-bb61fc734e95 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:02:51.095692 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f4a18-50af-4137-8f63-e06258b6698d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:02:51.096319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f4a18-50af-4137-8f63-e06258b6698d service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:02:51.096603 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f4a18-50af-4137-8f63-e06258b6698d service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:02:51.096873 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f4a18-50af-4137-8f63-e06258b6698d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:02:51.102644 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:02:51.106946 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f4a18-50af-4137-8f63-e06258b6698d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:02:51.108576 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f4a18-50af-4137-8f63-e06258b6698d service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:02:51.119719 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1349/2686] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:02:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:02:59.705605 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=8 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:03:01.115926 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:03:04.155220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41185da0-8a97-4ada-940a-8dc9b146ec00 service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:04.221179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41185da0-8a97-4ada-940a-8dc9b146ec00 service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:04.226099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5d5c2-8d29-4c98-a709-f39ef54eda60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:04.226755 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5d5c2-8d29-4c98-a709-f39ef54eda60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:04.227016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5d5c2-8d29-4c98-a709-f39ef54eda60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:04.227289 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5d5c2-8d29-4c98-a709-f39ef54eda60 admin admin] PATH_INFO: `/v3/users/7f2b3d80a68a4a34ae19056f4d63ce8b` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:04.228481 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5d5c2-8d29-4c98-a709-f39ef54eda60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2b3d80a68a4a34ae19056f4d63ce8b)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:04.230395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5d5c2-8d29-4c98-a709-f39ef54eda60 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:04.276704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-56c5d5c2-8d29-4c98-a709-f39ef54eda60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2b3d80a68a4a34ae19056f4d63ce8b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:04.284304 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-56c5d5c2-8d29-4c98-a709-f39ef54eda60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2b3d80a68a4a34ae19056f4d63ce8b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:04.297106 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-56c5d5c2-8d29-4c98-a709-f39ef54eda60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2b3d80a68a4a34ae19056f4d63ce8b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:04.303976 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-56c5d5c2-8d29-4c98-a709-f39ef54eda60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2b3d80a68a4a34ae19056f4d63ce8b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:04.304767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-56c5d5c2-8d29-4c98-a709-f39ef54eda60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2b3d80a68a4a34ae19056f4d63ce8b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:04.311216 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1338/2687] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:03:04 2022] DELETE /identity/v3/users/7f2b3d80a68a4a34ae19056f4d63ce8b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:03:04.489119 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f4a18-50af-4137-8f63-e06258b6698d service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:04.501138 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f4a18-50af-4137-8f63-e06258b6698d service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:04.560254 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb94c97-9a80-4811-9de9-490e9dd679f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:04.560927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb94c97-9a80-4811-9de9-490e9dd679f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:04.561215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb94c97-9a80-4811-9de9-490e9dd679f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:04.561493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb94c97-9a80-4811-9de9-490e9dd679f1 admin admin] PATH_INFO: `/v3/projects/7c6c0acf801848c9bdbc9bdc64d24866` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:04.562667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb94c97-9a80-4811-9de9-490e9dd679f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c6c0acf801848c9bdbc9bdc64d24866)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:04.564622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb94c97-9a80-4811-9de9-490e9dd679f1 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:04.611739 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ebb94c97-9a80-4811-9de9-490e9dd679f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c6c0acf801848c9bdbc9bdc64d24866', 'request_id': 'req-ebb94c97-9a80-4811-9de9-490e9dd679f1'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:04.617189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ebb94c97-9a80-4811-9de9-490e9dd679f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c6c0acf801848c9bdbc9bdc64d24866', 'request_id': 'req-ebb94c97-9a80-4811-9de9-490e9dd679f1'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:04.625755 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ebb94c97-9a80-4811-9de9-490e9dd679f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c6c0acf801848c9bdbc9bdc64d24866', 'request_id': 'req-ebb94c97-9a80-4811-9de9-490e9dd679f1'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:04.636901 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ebb94c97-9a80-4811-9de9-490e9dd679f1 admin admin] The token cache is being invalidate because project 7c6c0acf801848c9bdbc9bdc64d24866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:03:04.637193 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ebb94c97-9a80-4811-9de9-490e9dd679f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:04.638791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ebb94c97-9a80-4811-9de9-490e9dd679f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:04.640097 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1350/2688] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:03:04 2022] DELETE /identity/v3/projects/7c6c0acf801848c9bdbc9bdc64d24866 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 19:03:04.644003 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5d5c2-8d29-4c98-a709-f39ef54eda60 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:04.645432 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-56c5d5c2-8d29-4c98-a709-f39ef54eda60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:04.661622 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5d5c2-8d29-4c98-a709-f39ef54eda60 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:04.718695 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1aec20-d693-4e14-8b10-57ccd06e797c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:04.719334 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1aec20-d693-4e14-8b10-57ccd06e797c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:04.719616 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1aec20-d693-4e14-8b10-57ccd06e797c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:04.719888 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1aec20-d693-4e14-8b10-57ccd06e797c admin admin] PATH_INFO: `/v3/users/627207fc754d44d9afb7b3f0452d1ca0` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:04.721077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1aec20-d693-4e14-8b10-57ccd06e797c admin admin] RBAC: Authorizing `identity:delete_user(user_id=627207fc754d44d9afb7b3f0452d1ca0)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:04.722975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1aec20-d693-4e14-8b10-57ccd06e797c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:04.745119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:03:04.771121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4d1aec20-d693-4e14-8b10-57ccd06e797c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '627207fc754d44d9afb7b3f0452d1ca0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:04.779688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4d1aec20-d693-4e14-8b10-57ccd06e797c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '627207fc754d44d9afb7b3f0452d1ca0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:04.792020 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4d1aec20-d693-4e14-8b10-57ccd06e797c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '627207fc754d44d9afb7b3f0452d1ca0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:04.799026 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4d1aec20-d693-4e14-8b10-57ccd06e797c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '627207fc754d44d9afb7b3f0452d1ca0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:04.799696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4d1aec20-d693-4e14-8b10-57ccd06e797c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '627207fc754d44d9afb7b3f0452d1ca0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:04.805532 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1339/2689] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:03:04 2022] DELETE /identity/v3/users/627207fc754d44d9afb7b3f0452d1ca0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:03:04.985434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb94c97-9a80-4811-9de9-490e9dd679f1 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:04.986925 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ebb94c97-9a80-4811-9de9-490e9dd679f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:05.000156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb94c97-9a80-4811-9de9-490e9dd679f1 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:05.057784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d28d40-5017-45ff-90a6-5ec4743d9a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:05.058433 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d28d40-5017-45ff-90a6-5ec4743d9a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:05.058716 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d28d40-5017-45ff-90a6-5ec4743d9a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:05.058990 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d28d40-5017-45ff-90a6-5ec4743d9a05 admin admin] PATH_INFO: `/v3/projects/81ef73036f0d44c8bc1f5c1ac8d3abfd` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:05.060772 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d28d40-5017-45ff-90a6-5ec4743d9a05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ef73036f0d44c8bc1f5c1ac8d3abfd)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:05.062706 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d28d40-5017-45ff-90a6-5ec4743d9a05 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:05.097690 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:03:05.111000 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-79d28d40-5017-45ff-90a6-5ec4743d9a05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ef73036f0d44c8bc1f5c1ac8d3abfd', 'request_id': 'req-79d28d40-5017-45ff-90a6-5ec4743d9a05'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:05.116514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-79d28d40-5017-45ff-90a6-5ec4743d9a05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ef73036f0d44c8bc1f5c1ac8d3abfd', 'request_id': 'req-79d28d40-5017-45ff-90a6-5ec4743d9a05'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:05.126224 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-79d28d40-5017-45ff-90a6-5ec4743d9a05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ef73036f0d44c8bc1f5c1ac8d3abfd', 'request_id': 'req-79d28d40-5017-45ff-90a6-5ec4743d9a05'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:05.138592 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-79d28d40-5017-45ff-90a6-5ec4743d9a05 admin admin] The token cache is being invalidate because project 81ef73036f0d44c8bc1f5c1ac8d3abfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:03:05.138955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-79d28d40-5017-45ff-90a6-5ec4743d9a05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:05.139926 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-79d28d40-5017-45ff-90a6-5ec4743d9a05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:05.141138 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1351/2690] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:03:04 2022] DELETE /identity/v3/projects/81ef73036f0d44c8bc1f5c1ac8d3abfd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:03:05.146130 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795c8e3-afa8-4ff2-81ed-b0196aab116f None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:05.146425 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795c8e3-afa8-4ff2-81ed-b0196aab116f None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:05.146706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795c8e3-afa8-4ff2-81ed-b0196aab116f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:05.166209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-3795c8e3-afa8-4ff2-81ed-b0196aab116f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:03:05.167418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-3795c8e3-afa8-4ff2-81ed-b0196aab116f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:05.226573 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1340/2691] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:03:05.234891 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba258a-ff08-461f-9960-3dfb886a2bb9 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:05.235214 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba258a-ff08-461f-9960-3dfb886a2bb9 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:05.235531 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba258a-ff08-461f-9960-3dfb886a2bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:05.256125 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-d4ba258a-ff08-461f-9960-3dfb886a2bb9 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:03:05.257366 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d4ba258a-ff08-461f-9960-3dfb886a2bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:05.266021 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1352/2692] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:03:05.270345 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795c8e3-afa8-4ff2-81ed-b0196aab116f None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:05.280367 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795c8e3-afa8-4ff2-81ed-b0196aab116f None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:05.285088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925565c-805a-4e2a-b407-08e9c8328b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:05.285705 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925565c-805a-4e2a-b407-08e9c8328b84 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:05.285990 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925565c-805a-4e2a-b407-08e9c8328b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:05.286270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925565c-805a-4e2a-b407-08e9c8328b84 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:05.286972 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925565c-805a-4e2a-b407-08e9c8328b84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:03:05.287280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925565c-805a-4e2a-b407-08e9c8328b84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:05.288932 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925565c-805a-4e2a-b407-08e9c8328b84 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:05.301773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1341/2693] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:03:05.306216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba258a-ff08-461f-9960-3dfb886a2bb9 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:05.316244 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba258a-ff08-461f-9960-3dfb886a2bb9 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:05.320961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bc4ab-d60b-4654-88b1-6f8456d7e19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:05.321573 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bc4ab-d60b-4654-88b1-6f8456d7e19a admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:05.321851 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bc4ab-d60b-4654-88b1-6f8456d7e19a admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:05.322124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bc4ab-d60b-4654-88b1-6f8456d7e19a admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:05.322793 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bc4ab-d60b-4654-88b1-6f8456d7e19a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:05.324729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bc4ab-d60b-4654-88b1-6f8456d7e19a admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:05.351525 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1353/2694] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:03:05 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:03:05.355675 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925565c-805a-4e2a-b407-08e9c8328b84 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:05.367741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925565c-805a-4e2a-b407-08e9c8328b84 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:05.426990 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9117a2-6a21-42e4-b696-a5821892d08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:05.427649 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9117a2-6a21-42e4-b696-a5821892d08d admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:05.427936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9117a2-6a21-42e4-b696-a5821892d08d admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:05.428211 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9117a2-6a21-42e4-b696-a5821892d08d admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:05.428925 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9117a2-6a21-42e4-b696-a5821892d08d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:05.430855 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9117a2-6a21-42e4-b696-a5821892d08d admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:05.453645 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1342/2695] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:03:05 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:03:05.457528 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bc4ab-d60b-4654-88b1-6f8456d7e19a admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:05.468349 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bc4ab-d60b-4654-88b1-6f8456d7e19a admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:05.472932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e9c39-f104-4d27-a8aa-edd272f43546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:05.473569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e9c39-f104-4d27-a8aa-edd272f43546 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:05.473849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e9c39-f104-4d27-a8aa-edd272f43546 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:05.474121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e9c39-f104-4d27-a8aa-edd272f43546 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:05.474812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e9c39-f104-4d27-a8aa-edd272f43546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:05.476471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e9c39-f104-4d27-a8aa-edd272f43546 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:05.489035 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1354/2696] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:03:05.493042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9117a2-6a21-42e4-b696-a5821892d08d admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:05.504465 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9117a2-6a21-42e4-b696-a5821892d08d admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:05.509296 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9d00b-f8e4-4b14-922e-d4479aca67f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:05.509902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9d00b-f8e4-4b14-922e-d4479aca67f5 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:05.510178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9d00b-f8e4-4b14-922e-d4479aca67f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:05.510447 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9d00b-f8e4-4b14-922e-d4479aca67f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:05.511172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9d00b-f8e4-4b14-922e-d4479aca67f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:05.512875 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9d00b-f8e4-4b14-922e-d4479aca67f5 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:05.523943 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1343/2697] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 19:03:05.528331 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e9c39-f104-4d27-a8aa-edd272f43546 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:05.538839 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e9c39-f104-4d27-a8aa-edd272f43546 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:05.543488 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a119ee-9332-433d-9e52-0bf591e0f32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:05.544874 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a119ee-9332-433d-9e52-0bf591e0f32d admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:05.545095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a119ee-9332-433d-9e52-0bf591e0f32d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:05.545369 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a119ee-9332-433d-9e52-0bf591e0f32d admin admin] PATH_INFO: `/v3/projects/2f1552d9aec64d6cbf4a2bc34e76ceff/users/bc5ab67a2974483a9af7b9a6b0b18ce9/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:05.560779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a119ee-9332-433d-9e52-0bf591e0f32d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f1552d9aec64d6cbf4a2bc34e76ceff, user_id=bc5ab67a2974483a9af7b9a6b0b18ce9, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:05.563136 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a119ee-9332-433d-9e52-0bf591e0f32d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:05.577063 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1355/2698] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:03:05 2022] PUT /identity/v3/projects/2f1552d9aec64d6cbf4a2bc34e76ceff/users/bc5ab67a2974483a9af7b9a6b0b18ce9/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 19:03:05.588624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbd090-4ffa-4119-9962-7806d9a9dd7c None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:05.588909 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbd090-4ffa-4119-9962-7806d9a9dd7c None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:05.589184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbd090-4ffa-4119-9962-7806d9a9dd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:05.597188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:03:05.618781 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-23fbd090-4ffa-4119-9962-7806d9a9dd7c None None] MFA Rules not processed for user `bc5ab67a2974483a9af7b9a6b0b18ce9`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:03:05.619956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-23fbd090-4ffa-4119-9962-7806d9a9dd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:05.687282 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1344/2699] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:03:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:03:05.698989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a119ee-9332-433d-9e52-0bf591e0f32d admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:05.701902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-86a119ee-9332-433d-9e52-0bf591e0f32d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:05.781232 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a119ee-9332-433d-9e52-0bf591e0f32d admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:05.785190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9314e-4dbb-4c27-842e-3f75691943a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4799b0c9e5aa47d389121d2183666d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:05.785814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9314e-4dbb-4c27-842e-3f75691943a3 service glance] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:05.786092 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9314e-4dbb-4c27-842e-3f75691943a3 service glance] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:05.786363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9314e-4dbb-4c27-842e-3f75691943a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:05.796116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9314e-4dbb-4c27-842e-3f75691943a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:05.797591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9314e-4dbb-4c27-842e-3f75691943a3 service glance] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:05.807201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1356/2700] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:03:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:03:15.683702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=30 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:03:15.802848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=74,keystone:DELETE=5,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:03:52.915903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbd090-4ffa-4119-9962-7806d9a9dd7c None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:52.983869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbd090-4ffa-4119-9962-7806d9a9dd7c None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:52.988448 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0ecc4-211f-496a-9e17-07c76726e2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:52.989111 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0ecc4-211f-496a-9e17-07c76726e2a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:52.989391 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0ecc4-211f-496a-9e17-07c76726e2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:52.989664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0ecc4-211f-496a-9e17-07c76726e2a9 admin admin] PATH_INFO: `/v3/users/bc5ab67a2974483a9af7b9a6b0b18ce9` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:52.990923 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0ecc4-211f-496a-9e17-07c76726e2a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc5ab67a2974483a9af7b9a6b0b18ce9)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:52.992907 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0ecc4-211f-496a-9e17-07c76726e2a9 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:53.042061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-def0ecc4-211f-496a-9e17-07c76726e2a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc5ab67a2974483a9af7b9a6b0b18ce9'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:53.050127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-def0ecc4-211f-496a-9e17-07c76726e2a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc5ab67a2974483a9af7b9a6b0b18ce9'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:53.063334 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-def0ecc4-211f-496a-9e17-07c76726e2a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc5ab67a2974483a9af7b9a6b0b18ce9'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:53.071105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-def0ecc4-211f-496a-9e17-07c76726e2a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc5ab67a2974483a9af7b9a6b0b18ce9'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:53.071939 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-def0ecc4-211f-496a-9e17-07c76726e2a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc5ab67a2974483a9af7b9a6b0b18ce9'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:53.078828 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1345/2701] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:03:52 2022] DELETE /identity/v3/users/bc5ab67a2974483a9af7b9a6b0b18ce9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:03:53.083663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e98540-c9b1-4894-b6bb-700b83dc0864 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:53.084182 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e98540-c9b1-4894-b6bb-700b83dc0864 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:53.084330 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e98540-c9b1-4894-b6bb-700b83dc0864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:53.104541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-f1e98540-c9b1-4894-b6bb-700b83dc0864 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:03:53.105743 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f1e98540-c9b1-4894-b6bb-700b83dc0864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:53.170475 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1357/2702] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:03:53.180820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0ecc4-211f-496a-9e17-07c76726e2a9 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:53.182254 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-def0ecc4-211f-496a-9e17-07c76726e2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:53.241313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0ecc4-211f-496a-9e17-07c76726e2a9 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:53.245008 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd9736-a945-4d43-937c-fd38e0e3829c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:53.245632 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd9736-a945-4d43-937c-fd38e0e3829c service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:53.245917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd9736-a945-4d43-937c-fd38e0e3829c service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:53.246194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd9736-a945-4d43-937c-fd38e0e3829c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:53.255421 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd9736-a945-4d43-937c-fd38e0e3829c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:53.257158 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd9736-a945-4d43-937c-fd38e0e3829c service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:53.267468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1346/2703] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 19:03:53.548639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e98540-c9b1-4894-b6bb-700b83dc0864 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:53.550270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-f1e98540-c9b1-4894-b6bb-700b83dc0864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:53.566365 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e98540-c9b1-4894-b6bb-700b83dc0864 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:53.572411 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacac00-39cb-4384-9f2d-0cd6c4896ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:53.573170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacac00-39cb-4384-9f2d-0cd6c4896ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:53.573547 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacac00-39cb-4384-9f2d-0cd6c4896ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:53.573921 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacac00-39cb-4384-9f2d-0cd6c4896ffe admin admin] PATH_INFO: `/v3/projects/2f1552d9aec64d6cbf4a2bc34e76ceff` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:53.576068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceacac00-39cb-4384-9f2d-0cd6c4896ffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f1552d9aec64d6cbf4a2bc34e76ceff)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:53.578321 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceacac00-39cb-4384-9f2d-0cd6c4896ffe admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:53.630783 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ceacac00-39cb-4384-9f2d-0cd6c4896ffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f1552d9aec64d6cbf4a2bc34e76ceff', 'request_id': 'req-ceacac00-39cb-4384-9f2d-0cd6c4896ffe'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:53.636880 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ceacac00-39cb-4384-9f2d-0cd6c4896ffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f1552d9aec64d6cbf4a2bc34e76ceff', 'request_id': 'req-ceacac00-39cb-4384-9f2d-0cd6c4896ffe'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:53.646359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ceacac00-39cb-4384-9f2d-0cd6c4896ffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f1552d9aec64d6cbf4a2bc34e76ceff', 'request_id': 'req-ceacac00-39cb-4384-9f2d-0cd6c4896ffe'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:53.659726 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ceacac00-39cb-4384-9f2d-0cd6c4896ffe admin admin] The token cache is being invalidate because project 2f1552d9aec64d6cbf4a2bc34e76ceff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:03:53.660022 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ceacac00-39cb-4384-9f2d-0cd6c4896ffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:53.660868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ceacac00-39cb-4384-9f2d-0cd6c4896ffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:53.662745 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1358/2704] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:03:53 2022] DELETE /identity/v3/projects/2f1552d9aec64d6cbf4a2bc34e76ceff => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:03:53.667440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d60706-6c6e-4c91-83c9-91961dcd500e None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:53.667727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d60706-6c6e-4c91-83c9-91961dcd500e None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:53.668005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d60706-6c6e-4c91-83c9-91961dcd500e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:53.689905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-03d60706-6c6e-4c91-83c9-91961dcd500e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:03:53.691055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-03d60706-6c6e-4c91-83c9-91961dcd500e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:53.714889 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:03:53.764889 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1347/2705] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:03:53.772937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ffc29-c7c4-4081-a879-f60f42a03664 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:53.773227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ffc29-c7c4-4081-a879-f60f42a03664 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:53.773508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ffc29-c7c4-4081-a879-f60f42a03664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:53.794827 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-3d2ffc29-c7c4-4081-a879-f60f42a03664 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:03:53.796044 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-3d2ffc29-c7c4-4081-a879-f60f42a03664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:53.804080 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1359/2706] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:03:53.808449 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d60706-6c6e-4c91-83c9-91961dcd500e None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:53.818782 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d60706-6c6e-4c91-83c9-91961dcd500e None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:53.823361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4729eb-8edd-4140-907a-73883168dd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:53.823973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4729eb-8edd-4140-907a-73883168dd11 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:53.824256 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4729eb-8edd-4140-907a-73883168dd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:53.824564 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4729eb-8edd-4140-907a-73883168dd11 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:53.825359 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4729eb-8edd-4140-907a-73883168dd11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:03:53.825678 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4729eb-8edd-4140-907a-73883168dd11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:53.827307 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4729eb-8edd-4140-907a-73883168dd11 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:53.839586 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1348/2707] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:03:53.844725 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ffc29-c7c4-4081-a879-f60f42a03664 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:53.854308 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ffc29-c7c4-4081-a879-f60f42a03664 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:53.859766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb46986-6d2f-4c67-a7e3-631909609a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:53.860347 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb46986-6d2f-4c67-a7e3-631909609a30 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:53.860633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb46986-6d2f-4c67-a7e3-631909609a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:53.860905 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb46986-6d2f-4c67-a7e3-631909609a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:53.861584 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb46986-6d2f-4c67-a7e3-631909609a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:53.863492 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb46986-6d2f-4c67-a7e3-631909609a30 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:53.893405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1360/2708] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:03:53 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 19:03:53.897466 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4729eb-8edd-4140-907a-73883168dd11 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:53.907549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4729eb-8edd-4140-907a-73883168dd11 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:53.964580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd17d48-4567-43a4-8d83-db5b690f86b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:53.965187 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd17d48-4567-43a4-8d83-db5b690f86b8 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:53.965526 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd17d48-4567-43a4-8d83-db5b690f86b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:53.965804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd17d48-4567-43a4-8d83-db5b690f86b8 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:53.966630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd17d48-4567-43a4-8d83-db5b690f86b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:53.968574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd17d48-4567-43a4-8d83-db5b690f86b8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:53.989704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1349/2709] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:03:53 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:03:53.993770 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb46986-6d2f-4c67-a7e3-631909609a30 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:54.004927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb46986-6d2f-4c67-a7e3-631909609a30 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:54.009878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1b6b5-2bf5-4ad1-98a4-baa59b3a3c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:54.010485 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1b6b5-2bf5-4ad1-98a4-baa59b3a3c79 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:54.010765 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1b6b5-2bf5-4ad1-98a4-baa59b3a3c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:54.011038 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1b6b5-2bf5-4ad1-98a4-baa59b3a3c79 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:54.011735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1b6b5-2bf5-4ad1-98a4-baa59b3a3c79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:54.014032 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1b6b5-2bf5-4ad1-98a4-baa59b3a3c79 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:54.023879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1361/2710] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:03:54.027704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd17d48-4567-43a4-8d83-db5b690f86b8 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:54.038957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd17d48-4567-43a4-8d83-db5b690f86b8 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:54.043630 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da777c6f-16fd-4998-b413-2c91c5593b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:54.045095 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da777c6f-16fd-4998-b413-2c91c5593b38 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:54.045381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da777c6f-16fd-4998-b413-2c91c5593b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:54.045657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da777c6f-16fd-4998-b413-2c91c5593b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:54.046345 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da777c6f-16fd-4998-b413-2c91c5593b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:54.047919 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da777c6f-16fd-4998-b413-2c91c5593b38 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:54.057700 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1350/2711] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:03:54.061814 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1b6b5-2bf5-4ad1-98a4-baa59b3a3c79 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:54.072085 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1b6b5-2bf5-4ad1-98a4-baa59b3a3c79 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:54.078233 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666c3bf-c491-4d66-b46a-fabb6066fb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:54.078961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666c3bf-c491-4d66-b46a-fabb6066fb8d admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:54.079300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666c3bf-c491-4d66-b46a-fabb6066fb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:54.079609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666c3bf-c491-4d66-b46a-fabb6066fb8d admin admin] PATH_INFO: `/v3/projects/385c8f77bee14271bb8d12f2eb650fc4/users/5664328d840946478fd2b2047132d911/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:54.095174 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666c3bf-c491-4d66-b46a-fabb6066fb8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=385c8f77bee14271bb8d12f2eb650fc4, user_id=5664328d840946478fd2b2047132d911, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:54.097870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666c3bf-c491-4d66-b46a-fabb6066fb8d admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:54.111185 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1362/2712] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:03:54 2022] PUT /identity/v3/projects/385c8f77bee14271bb8d12f2eb650fc4/users/5664328d840946478fd2b2047132d911/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:03:54.121012 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a9564-76c8-426a-b5f4-2e4298c1c148 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:54.121300 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a9564-76c8-426a-b5f4-2e4298c1c148 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:54.121602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a9564-76c8-426a-b5f4-2e4298c1c148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:54.151025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-371a9564-76c8-426a-b5f4-2e4298c1c148 None None] MFA Rules not processed for user `5664328d840946478fd2b2047132d911`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:03:54.152248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-371a9564-76c8-426a-b5f4-2e4298c1c148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:54.218917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1351/2713] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:03:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:03:54.230216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666c3bf-c491-4d66-b46a-fabb6066fb8d admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:54.232310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c666c3bf-c491-4d66-b46a-fabb6066fb8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:54.255063 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:03:54.293459 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666c3bf-c491-4d66-b46a-fabb6066fb8d admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:54.297638 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4ed1f-da04-4032-8859-df406b90202e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:54.298275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4ed1f-da04-4032-8859-df406b90202e service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:54.298590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4ed1f-da04-4032-8859-df406b90202e service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:54.298934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4ed1f-da04-4032-8859-df406b90202e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:54.307818 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4ed1f-da04-4032-8859-df406b90202e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:54.310019 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4ed1f-da04-4032-8859-df406b90202e service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:54.319598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1363/2714] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:03:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:03:54.347528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a9564-76c8-426a-b5f4-2e4298c1c148 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:54.366594 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:03:54.412639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a9564-76c8-426a-b5f4-2e4298c1c148 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:54.417213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfe252-95fe-4880-9d96-08fe78dc2756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:54.417852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfe252-95fe-4880-9d96-08fe78dc2756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:54.418133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfe252-95fe-4880-9d96-08fe78dc2756 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:54.418409 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfe252-95fe-4880-9d96-08fe78dc2756 admin admin] PATH_INFO: `/v3/users/5664328d840946478fd2b2047132d911` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:54.419660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebfe252-95fe-4880-9d96-08fe78dc2756 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5664328d840946478fd2b2047132d911)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:54.421606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebfe252-95fe-4880-9d96-08fe78dc2756 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:54.467906 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-eebfe252-95fe-4880-9d96-08fe78dc2756 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5664328d840946478fd2b2047132d911'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:54.474756 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-eebfe252-95fe-4880-9d96-08fe78dc2756 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5664328d840946478fd2b2047132d911'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:54.487574 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-eebfe252-95fe-4880-9d96-08fe78dc2756 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5664328d840946478fd2b2047132d911'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:54.494316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-eebfe252-95fe-4880-9d96-08fe78dc2756 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5664328d840946478fd2b2047132d911'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:54.494967 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-eebfe252-95fe-4880-9d96-08fe78dc2756 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5664328d840946478fd2b2047132d911'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:54.501805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1352/2715] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:03:54 2022] DELETE /identity/v3/users/5664328d840946478fd2b2047132d911 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:03:54.506615 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cd492-e9c8-4cac-b94b-34794acbdfa6 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:54.506908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cd492-e9c8-4cac-b94b-34794acbdfa6 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:54.507185 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cd492-e9c8-4cac-b94b-34794acbdfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:54.526742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-c52cd492-e9c8-4cac-b94b-34794acbdfa6 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:03:54.527963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c52cd492-e9c8-4cac-b94b-34794acbdfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:54.587029 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1364/2716] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:03:54.597938 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfe252-95fe-4880-9d96-08fe78dc2756 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:54.599482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-eebfe252-95fe-4880-9d96-08fe78dc2756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:54.661146 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfe252-95fe-4880-9d96-08fe78dc2756 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:54.664852 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0b8ef-34cc-4c1d-b6c5-2fb6c448cbfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:54.665472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0b8ef-34cc-4c1d-b6c5-2fb6c448cbfc service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:54.665756 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0b8ef-34cc-4c1d-b6c5-2fb6c448cbfc service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:54.666030 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0b8ef-34cc-4c1d-b6c5-2fb6c448cbfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:54.677687 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0b8ef-34cc-4c1d-b6c5-2fb6c448cbfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:54.679125 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0b8ef-34cc-4c1d-b6c5-2fb6c448cbfc service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:54.690310 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1353/2717] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:03:54.963514 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cd492-e9c8-4cac-b94b-34794acbdfa6 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:54.965013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c52cd492-e9c8-4cac-b94b-34794acbdfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:54.978850 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cd492-e9c8-4cac-b94b-34794acbdfa6 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:54.984487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46279fc5-8854-4ad8-9e71-b8114a329837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:54.985189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279fc5-8854-4ad8-9e71-b8114a329837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:54.985481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279fc5-8854-4ad8-9e71-b8114a329837 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:54.985755 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279fc5-8854-4ad8-9e71-b8114a329837 admin admin] PATH_INFO: `/v3/projects/385c8f77bee14271bb8d12f2eb650fc4` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:54.986947 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46279fc5-8854-4ad8-9e71-b8114a329837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=385c8f77bee14271bb8d12f2eb650fc4)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:54.988951 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46279fc5-8854-4ad8-9e71-b8114a329837 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:55.037866 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-46279fc5-8854-4ad8-9e71-b8114a329837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '385c8f77bee14271bb8d12f2eb650fc4', 'request_id': 'req-46279fc5-8854-4ad8-9e71-b8114a329837'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:55.043180 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-46279fc5-8854-4ad8-9e71-b8114a329837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '385c8f77bee14271bb8d12f2eb650fc4', 'request_id': 'req-46279fc5-8854-4ad8-9e71-b8114a329837'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:55.050927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-46279fc5-8854-4ad8-9e71-b8114a329837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '385c8f77bee14271bb8d12f2eb650fc4', 'request_id': 'req-46279fc5-8854-4ad8-9e71-b8114a329837'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:55.062568 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-46279fc5-8854-4ad8-9e71-b8114a329837 admin admin] The token cache is being invalidate because project 385c8f77bee14271bb8d12f2eb650fc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:03:55.062887 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-46279fc5-8854-4ad8-9e71-b8114a329837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:55.063523 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-46279fc5-8854-4ad8-9e71-b8114a329837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:03:55.065061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1365/2718] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:03:54 2022] DELETE /identity/v3/projects/385c8f77bee14271bb8d12f2eb650fc4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:03:55.071507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6880881-41ed-4df6-9e16-61d75ab4bbec None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:55.071976 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6880881-41ed-4df6-9e16-61d75ab4bbec None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:55.072089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6880881-41ed-4df6-9e16-61d75ab4bbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:55.092257 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-e6880881-41ed-4df6-9e16-61d75ab4bbec None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:03:55.093475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e6880881-41ed-4df6-9e16-61d75ab4bbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:55.127534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:03:55.155313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1354/2719] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:03:55.163301 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac2163-61de-4562-924f-c9f05660f72e None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:55.163784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac2163-61de-4562-924f-c9f05660f72e None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:55.163911 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac2163-61de-4562-924f-c9f05660f72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:55.185450 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-c4ac2163-61de-4562-924f-c9f05660f72e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:03:55.186659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-c4ac2163-61de-4562-924f-c9f05660f72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:55.195540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1366/2720] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:03:55.200606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6880881-41ed-4df6-9e16-61d75ab4bbec None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:55.211299 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6880881-41ed-4df6-9e16-61d75ab4bbec None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:55.216917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b5c45-9a91-4681-b67e-a6973c716e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:55.217531 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b5c45-9a91-4681-b67e-a6973c716e86 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:55.217925 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b5c45-9a91-4681-b67e-a6973c716e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:55.218209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b5c45-9a91-4681-b67e-a6973c716e86 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:55.218914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b5c45-9a91-4681-b67e-a6973c716e86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:03:55.219218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b5c45-9a91-4681-b67e-a6973c716e86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:55.221142 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b5c45-9a91-4681-b67e-a6973c716e86 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:55.232894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1355/2721] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:03:55.237343 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac2163-61de-4562-924f-c9f05660f72e None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:55.246872 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac2163-61de-4562-924f-c9f05660f72e None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:55.251606 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188b448-3dc4-48b8-a51c-fa13cd893aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:55.252352 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188b448-3dc4-48b8-a51c-fa13cd893aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:55.252639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188b448-3dc4-48b8-a51c-fa13cd893aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:55.252916 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188b448-3dc4-48b8-a51c-fa13cd893aa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:55.253589 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c188b448-3dc4-48b8-a51c-fa13cd893aa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:55.255493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c188b448-3dc4-48b8-a51c-fa13cd893aa0 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:55.284395 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1367/2722] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:03:55 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 19:03:55.288537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b5c45-9a91-4681-b67e-a6973c716e86 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:55.299075 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b5c45-9a91-4681-b67e-a6973c716e86 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:55.356917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c24b1-130e-4d0f-a15b-0e749b250883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:55.357539 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c24b1-130e-4d0f-a15b-0e749b250883 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:55.357823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c24b1-130e-4d0f-a15b-0e749b250883 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:55.358101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c24b1-130e-4d0f-a15b-0e749b250883 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:55.358777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c24b1-130e-4d0f-a15b-0e749b250883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:55.360732 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c24b1-130e-4d0f-a15b-0e749b250883 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:55.383639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1356/2723] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:03:55 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:03:55.387738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188b448-3dc4-48b8-a51c-fa13cd893aa0 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:55.398187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188b448-3dc4-48b8-a51c-fa13cd893aa0 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:55.402738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb02c9-9c6d-43c2-a6fc-6f5210874416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:55.403359 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb02c9-9c6d-43c2-a6fc-6f5210874416 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:55.403749 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb02c9-9c6d-43c2-a6fc-6f5210874416 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:55.403907 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb02c9-9c6d-43c2-a6fc-6f5210874416 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:55.404675 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb02c9-9c6d-43c2-a6fc-6f5210874416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:55.406288 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb02c9-9c6d-43c2-a6fc-6f5210874416 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:55.416420 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1368/2724] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:03:55.421020 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c24b1-130e-4d0f-a15b-0e749b250883 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:55.431031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c24b1-130e-4d0f-a15b-0e749b250883 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:55.436453 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846498a-146a-44f5-847a-23dcc323a4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:55.437074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846498a-146a-44f5-847a-23dcc323a4ce admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:55.437362 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846498a-146a-44f5-847a-23dcc323a4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:55.437643 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846498a-146a-44f5-847a-23dcc323a4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:55.438350 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846498a-146a-44f5-847a-23dcc323a4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:55.439950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846498a-146a-44f5-847a-23dcc323a4ce admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:55.452242 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1357/2725] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:03:55.456600 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb02c9-9c6d-43c2-a6fc-6f5210874416 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:55.466719 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb02c9-9c6d-43c2-a6fc-6f5210874416 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:55.471294 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee6fba-1f27-4f7c-8db7-0abf36da2304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:55.471952 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee6fba-1f27-4f7c-8db7-0abf36da2304 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:55.472235 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee6fba-1f27-4f7c-8db7-0abf36da2304 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:55.472616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee6fba-1f27-4f7c-8db7-0abf36da2304 admin admin] PATH_INFO: `/v3/projects/7041552fa8cc4a11ae098f1d2a16e359/users/4f7e62d82d054b98bd22dc6958577a74/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:55.474823 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:03:55.492443 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee6fba-1f27-4f7c-8db7-0abf36da2304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7041552fa8cc4a11ae098f1d2a16e359, user_id=4f7e62d82d054b98bd22dc6958577a74, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:55.495124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee6fba-1f27-4f7c-8db7-0abf36da2304 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:55.509730 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1369/2726] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:03:55 2022] PUT /identity/v3/projects/7041552fa8cc4a11ae098f1d2a16e359/users/4f7e62d82d054b98bd22dc6958577a74/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:03:55.515079 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d670a1-b82d-4250-925d-fd9a4cba1420 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:55.515369 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d670a1-b82d-4250-925d-fd9a4cba1420 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:55.515638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d670a1-b82d-4250-925d-fd9a4cba1420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:55.534905 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-77d670a1-b82d-4250-925d-fd9a4cba1420 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:03:55.536102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-77d670a1-b82d-4250-925d-fd9a4cba1420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:55.596904 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1358/2727] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:03:55.606565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee6fba-1f27-4f7c-8db7-0abf36da2304 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:55.608530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-faee6fba-1f27-4f7c-8db7-0abf36da2304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:55.667720 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee6fba-1f27-4f7c-8db7-0abf36da2304 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:55.671667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec14d1-3471-49e9-bf23-d64fa714dbdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:55.672313 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec14d1-3471-49e9-bf23-d64fa714dbdd service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:55.672604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec14d1-3471-49e9-bf23-d64fa714dbdd service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:55.672883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec14d1-3471-49e9-bf23-d64fa714dbdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:55.682167 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec14d1-3471-49e9-bf23-d64fa714dbdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:55.683618 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec14d1-3471-49e9-bf23-d64fa714dbdd service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:55.694404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1370/2728] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:03:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:03:59.706636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684aa38-0fa3-49e4-9951-292357e62add None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:59.707162 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684aa38-0fa3-49e4-9951-292357e62add None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:59.707276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684aa38-0fa3-49e4-9951-292357e62add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:59.737536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-0684aa38-0fa3-49e4-9951-292357e62add None None] MFA Rules not processed for user `4f7e62d82d054b98bd22dc6958577a74`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:03:59.738760 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-0684aa38-0fa3-49e4-9951-292357e62add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:03:59.784082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:03:59.846866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1359/2729] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:03:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:03:59.851465 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec14d1-3471-49e9-bf23-d64fa714dbdd service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:59.865306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec14d1-3471-49e9-bf23-d64fa714dbdd service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:59.870124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc56b3-7bae-4358-808b-6e89ae87d1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:59.870828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc56b3-7bae-4358-808b-6e89ae87d1a0 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:59.871201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc56b3-7bae-4358-808b-6e89ae87d1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:59.871575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc56b3-7bae-4358-808b-6e89ae87d1a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:59.872545 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc56b3-7bae-4358-808b-6e89ae87d1a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:59.874418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc56b3-7bae-4358-808b-6e89ae87d1a0 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:03:59.905910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1371/2730] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:03:59 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:03:59.910147 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684aa38-0fa3-49e4-9951-292357e62add None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:03:59.921900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684aa38-0fa3-49e4-9951-292357e62add None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:03:59.992535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b0613-e5be-4ffb-8059-a84cc299a0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:03:59.993180 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b0613-e5be-4ffb-8059-a84cc299a0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:03:59.993466 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b0613-e5be-4ffb-8059-a84cc299a0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:03:59.993744 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b0613-e5be-4ffb-8059-a84cc299a0e9 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:03:59.994425 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b0613-e5be-4ffb-8059-a84cc299a0e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:03:59.996390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b0613-e5be-4ffb-8059-a84cc299a0e9 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:04:00.020194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1360/2731] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:03:59 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:04:00.024237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc56b3-7bae-4358-808b-6e89ae87d1a0 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:04:00.035652 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc56b3-7bae-4358-808b-6e89ae87d1a0 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:04:00.040580 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e8a12-4b1e-415d-9f9a-d1951c70f567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:04:00.041219 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e8a12-4b1e-415d-9f9a-d1951c70f567 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:04:00.041502 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e8a12-4b1e-415d-9f9a-d1951c70f567 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:04:00.041777 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e8a12-4b1e-415d-9f9a-d1951c70f567 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:04:00.042483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e8a12-4b1e-415d-9f9a-d1951c70f567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:04:00.044371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e8a12-4b1e-415d-9f9a-d1951c70f567 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:04:00.060132 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1372/2732] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:04:00.063944 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b0613-e5be-4ffb-8059-a84cc299a0e9 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:04:00.076115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b0613-e5be-4ffb-8059-a84cc299a0e9 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:04:00.081761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9640e3-74e5-4a4c-b67f-c075baf8e200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:04:00.082457 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9640e3-74e5-4a4c-b67f-c075baf8e200 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:04:00.082742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9640e3-74e5-4a4c-b67f-c075baf8e200 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:04:00.083016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9640e3-74e5-4a4c-b67f-c075baf8e200 admin admin] PATH_INFO: `/v3/projects/81fce08714e64e22a0c6466499829fdb/users/073c1605dd43489d83472c7523d65bf9/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:04:00.102637 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9640e3-74e5-4a4c-b67f-c075baf8e200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81fce08714e64e22a0c6466499829fdb, user_id=073c1605dd43489d83472c7523d65bf9, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:04:00.105406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9640e3-74e5-4a4c-b67f-c075baf8e200 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:04:00.120585 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1361/2733] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:04:00 2022] PUT /identity/v3/projects/81fce08714e64e22a0c6466499829fdb/users/073c1605dd43489d83472c7523d65bf9/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:04:00.125801 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e8a12-4b1e-415d-9f9a-d1951c70f567 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:04:00.138960 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e8a12-4b1e-415d-9f9a-d1951c70f567 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:04:00.207846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074890bf-72ab-4306-9ac3-c62555465b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:04:00.208908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074890bf-72ab-4306-9ac3-c62555465b08 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:04:00.209301 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074890bf-72ab-4306-9ac3-c62555465b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:04:00.209740 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074890bf-72ab-4306-9ac3-c62555465b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:04:00.210988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074890bf-72ab-4306-9ac3-c62555465b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:04:00.213554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074890bf-72ab-4306-9ac3-c62555465b08 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:04:00.226697 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1373/2734] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:04:00.230858 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9640e3-74e5-4a4c-b67f-c075baf8e200 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:04:00.242444 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9640e3-74e5-4a4c-b67f-c075baf8e200 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:04:00.247383 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c056a-d40a-458e-ab53-9ab4a5e8c27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:04:00.248220 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c056a-d40a-458e-ab53-9ab4a5e8c27e admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:04:00.248540 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c056a-d40a-458e-ab53-9ab4a5e8c27e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:04:00.248820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c056a-d40a-458e-ab53-9ab4a5e8c27e admin admin] PATH_INFO: `/v3/projects/81fce08714e64e22a0c6466499829fdb/users/073c1605dd43489d83472c7523d65bf9/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:04:00.250955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c056a-d40a-458e-ab53-9ab4a5e8c27e admin admin] RBAC: Authorizing `identity:create_grant(project_id=81fce08714e64e22a0c6466499829fdb, user_id=073c1605dd43489d83472c7523d65bf9, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:04:00.253793 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c056a-d40a-458e-ab53-9ab4a5e8c27e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:04:00.266070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1362/2735] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:04:00 2022] PUT /identity/v3/projects/81fce08714e64e22a0c6466499829fdb/users/073c1605dd43489d83472c7523d65bf9/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:04:03.273532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642d549-7b68-41f0-bd2c-e9f412e6cbad None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:04:03.274009 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642d549-7b68-41f0-bd2c-e9f412e6cbad None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:04:03.274111 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642d549-7b68-41f0-bd2c-e9f412e6cbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:04:03.303457 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-1642d549-7b68-41f0-bd2c-e9f412e6cbad None None] MFA Rules not processed for user `cd2bc7050bb9413c8fddd9c7864f537a`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:04:03.304973 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1642d549-7b68-41f0-bd2c-e9f412e6cbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:04:03.374097 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:04:03.395174 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1374/2736] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:04:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:04:04.248809 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c056a-d40a-458e-ab53-9ab4a5e8c27e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:04:04.333363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c056a-d40a-458e-ab53-9ab4a5e8c27e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:04:04.337461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6b520-bdc2-431c-9262-95f3c2bee0e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:04:04.338096 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6b520-bdc2-431c-9262-95f3c2bee0e9 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:04:04.338383 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6b520-bdc2-431c-9262-95f3c2bee0e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:04:04.338664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6b520-bdc2-431c-9262-95f3c2bee0e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:04:04.340556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a3d6b520-bdc2-431c-9262-95f3c2bee0e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:04:04.356945 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6b520-bdc2-431c-9262-95f3c2bee0e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:04:04.358414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6b520-bdc2-431c-9262-95f3c2bee0e9 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:04:04.368242 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1363/2737] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:04:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:04:05.588568 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642d549-7b68-41f0-bd2c-e9f412e6cbad None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:04:05.591434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-1642d549-7b68-41f0-bd2c-e9f412e6cbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:04:05.698378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642d549-7b68-41f0-bd2c-e9f412e6cbad None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:04:05.705639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87110cc5-e596-4513-bea0-3d9663fad79f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:04:05.706663 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87110cc5-e596-4513-bea0-3d9663fad79f service placement] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:04:05.707046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87110cc5-e596-4513-bea0-3d9663fad79f service placement] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:04:05.707416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87110cc5-e596-4513-bea0-3d9663fad79f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:04:05.710434 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-87110cc5-e596-4513-bea0-3d9663fad79f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:04:05.725651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87110cc5-e596-4513-bea0-3d9663fad79f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:04:05.727988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87110cc5-e596-4513-bea0-3d9663fad79f service placement] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:04:05.741355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1375/2738] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:04:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:04:05.785038 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbdfcb-f1fe-43de-9591-102e3c91c2e6 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:04:05.786034 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbdfcb-f1fe-43de-9591-102e3c91c2e6 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:04:05.786412 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbdfcb-f1fe-43de-9591-102e3c91c2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:04:05.819623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-5bfbdfcb-f1fe-43de-9591-102e3c91c2e6 None None] MFA Rules not processed for user `073c1605dd43489d83472c7523d65bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:04:05.821086 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-5bfbdfcb-f1fe-43de-9591-102e3c91c2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:04:05.843478 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:04:05.916580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1364/2739] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:04:05.936127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87110cc5-e596-4513-bea0-3d9663fad79f service placement] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:04:05.937314 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-87110cc5-e596-4513-bea0-3d9663fad79f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:04:06.019443 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87110cc5-e596-4513-bea0-3d9663fad79f service placement] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:04:06.025322 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1451930-08d4-4b98-bbea-2405168765f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:04:06.026095 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1451930-08d4-4b98-bbea-2405168765f1 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:04:06.026289 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1451930-08d4-4b98-bbea-2405168765f1 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:04:06.026526 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1451930-08d4-4b98-bbea-2405168765f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:04:06.097759 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1451930-08d4-4b98-bbea-2405168765f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:04:06.099393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1451930-08d4-4b98-bbea-2405168765f1 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:04:06.116428 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1376/2740] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:04:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:04:08.275640 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbdfcb-f1fe-43de-9591-102e3c91c2e6 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:04:08.304202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbdfcb-f1fe-43de-9591-102e3c91c2e6 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:04:08.324535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f7b9b-be49-4338-ad11-d86e05786a97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:04:08.328664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f7b9b-be49-4338-ad11-d86e05786a97 service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:04:08.329040 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f7b9b-be49-4338-ad11-d86e05786a97 service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:04:08.329415 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f7b9b-be49-4338-ad11-d86e05786a97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:04:08.338666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f65f7b9b-be49-4338-ad11-d86e05786a97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:04:08.383275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f7b9b-be49-4338-ad11-d86e05786a97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:04:08.383275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f7b9b-be49-4338-ad11-d86e05786a97 service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:04:08.444119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1365/2741] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:04:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 19:04:16.104480 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=91 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:04:18.433689 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=26 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:04:31.749872 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c770a2-3896-4152-b5b9-ddb39d5a063d None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:04:31.750450 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c770a2-3896-4152-b5b9-ddb39d5a063d None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:04:31.750450 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c770a2-3896-4152-b5b9-ddb39d5a063d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:04:31.771275 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-59c770a2-3896-4152-b5b9-ddb39d5a063d None None] MFA Rules not processed for user `4f7e62d82d054b98bd22dc6958577a74`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:04:31.772577 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-59c770a2-3896-4152-b5b9-ddb39d5a063d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:04:31.780438 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1377/2742] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:04:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:04:32.115559 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f7b9b-be49-4338-ad11-d86e05786a97 service nova] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:04:32.125784 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f7b9b-be49-4338-ad11-d86e05786a97 service nova] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:04:32.129429 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2437676-3c41-4a5f-a70b-99fc80bae94c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:04:32.130042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2437676-3c41-4a5f-a70b-99fc80bae94c service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:04:32.130327 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2437676-3c41-4a5f-a70b-99fc80bae94c service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:04:32.130604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2437676-3c41-4a5f-a70b-99fc80bae94c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:04:32.132685 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-f2437676-3c41-4a5f-a70b-99fc80bae94c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:04:32.144277 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2437676-3c41-4a5f-a70b-99fc80bae94c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:04:32.145769 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2437676-3c41-4a5f-a70b-99fc80bae94c service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:04:32.155953 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1366/2743] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:04:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:04:41.765200 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:04:42.150475 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=6 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:04:44.200999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c770a2-3896-4152-b5b9-ddb39d5a063d None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:04:44.202443 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-59c770a2-3896-4152-b5b9-ddb39d5a063d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:04:44.214551 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c770a2-3896-4152-b5b9-ddb39d5a063d None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:04:44.218834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f48e4-62bb-4358-a035-f30ccb4d8901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:04:44.219484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f48e4-62bb-4358-a035-f30ccb4d8901 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:04:44.219742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f48e4-62bb-4358-a035-f30ccb4d8901 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:04:44.220016 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f48e4-62bb-4358-a035-f30ccb4d8901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:04:44.229545 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f48e4-62bb-4358-a035-f30ccb4d8901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:04:44.230945 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f48e4-62bb-4358-a035-f30ccb4d8901 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:04:44.240964 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1378/2744] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:04:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 19:04:47.922218 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2437676-3c41-4a5f-a70b-99fc80bae94c service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:04:47.930888 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2437676-3c41-4a5f-a70b-99fc80bae94c service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:04:47.937332 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0a6a6-95c4-4ae4-a39b-7fa554329ea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:04:47.937971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0a6a6-95c4-4ae4-a39b-7fa554329ea5 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:04:47.938280 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0a6a6-95c4-4ae4-a39b-7fa554329ea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:04:47.938548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0a6a6-95c4-4ae4-a39b-7fa554329ea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:04:47.940374 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e5b0a6a6-95c4-4ae4-a39b-7fa554329ea5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:04:47.953270 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0a6a6-95c4-4ae4-a39b-7fa554329ea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:04:47.954855 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0a6a6-95c4-4ae4-a39b-7fa554329ea5 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:04:47.964049 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1367/2745] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:04:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:04:48.038245 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f48e4-62bb-4358-a035-f30ccb4d8901 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:04:48.047561 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f48e4-62bb-4358-a035-f30ccb4d8901 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:04:48.059855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1177c-6cf2-4dd3-923b-19656bbdae20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:04:48.059855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1177c-6cf2-4dd3-923b-19656bbdae20 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:04:48.059855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1177c-6cf2-4dd3-923b-19656bbdae20 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:04:48.059855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1177c-6cf2-4dd3-923b-19656bbdae20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:04:48.060453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-77e1177c-6cf2-4dd3-923b-19656bbdae20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:04:48.069536 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1177c-6cf2-4dd3-923b-19656bbdae20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:04:48.070959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1177c-6cf2-4dd3-923b-19656bbdae20 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:04:48.082310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1379/2746] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:04:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:04:57.960085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=6 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:04:58.077618 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=12 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:05:46.907654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0a6a6-95c4-4ae4-a39b-7fa554329ea5 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:05:46.909078 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-e5b0a6a6-95c4-4ae4-a39b-7fa554329ea5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:05:46.964264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0a6a6-95c4-4ae4-a39b-7fa554329ea5 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:05:46.968777 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2927e6-2945-4ef8-b80c-53d930838ad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:05:46.969553 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2927e6-2945-4ef8-b80c-53d930838ad8 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:05:46.969840 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2927e6-2945-4ef8-b80c-53d930838ad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:05:46.970115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2927e6-2945-4ef8-b80c-53d930838ad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:05:46.971864 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-bc2927e6-2945-4ef8-b80c-53d930838ad8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:05:46.985393 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2927e6-2945-4ef8-b80c-53d930838ad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:05:46.986757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2927e6-2945-4ef8-b80c-53d930838ad8 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:05:46.997853 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1368/2747] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:05:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:05:48.405991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1177c-6cf2-4dd3-923b-19656bbdae20 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:05:48.414757 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1177c-6cf2-4dd3-923b-19656bbdae20 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:05:48.419061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e17313-444b-4eae-9d67-459565ee62e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:05:48.419649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e17313-444b-4eae-9d67-459565ee62e0 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:05:48.419935 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e17313-444b-4eae-9d67-459565ee62e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:05:48.420206 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e17313-444b-4eae-9d67-459565ee62e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:05:48.422008 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-03e17313-444b-4eae-9d67-459565ee62e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:05:48.436713 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e17313-444b-4eae-9d67-459565ee62e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:05:48.438145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e17313-444b-4eae-9d67-459565ee62e0 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:05:48.449277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1380/2748] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:05:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:05:56.992934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=26 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:05:58.444450 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=6 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:06:00.003513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff49233-533c-411b-a185-04b1fc5a1227 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:06:00.004017 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff49233-533c-411b-a185-04b1fc5a1227 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:06:00.004214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff49233-533c-411b-a185-04b1fc5a1227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:06:00.025264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-fff49233-533c-411b-a185-04b1fc5a1227 None None] MFA Rules not processed for user `cd2bc7050bb9413c8fddd9c7864f537a`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:06:00.026482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-fff49233-533c-411b-a185-04b1fc5a1227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:06:00.033691 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1369/2749] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:06:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:06:02.555413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e17313-444b-4eae-9d67-459565ee62e0 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:06:02.564882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e17313-444b-4eae-9d67-459565ee62e0 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:06:02.568870 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b12fd2-e8a5-4aa2-acb0-4a496ec7f49b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:06:02.569587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b12fd2-e8a5-4aa2-acb0-4a496ec7f49b service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:06:02.570089 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b12fd2-e8a5-4aa2-acb0-4a496ec7f49b service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:06:02.570594 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b12fd2-e8a5-4aa2-acb0-4a496ec7f49b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:06:02.572649 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-55b12fd2-e8a5-4aa2-acb0-4a496ec7f49b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:06:02.584433 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b12fd2-e8a5-4aa2-acb0-4a496ec7f49b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:06:02.586015 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b12fd2-e8a5-4aa2-acb0-4a496ec7f49b service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:06:02.595936 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1381/2750] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:06:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:06:10.020455 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:06:12.591836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=6 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:06:41.919932 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff49233-533c-411b-a185-04b1fc5a1227 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:06:41.929805 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff49233-533c-411b-a185-04b1fc5a1227 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:06:41.933603 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9486c8a-21a4-4315-91c4-105b46489440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:06:41.934225 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9486c8a-21a4-4315-91c4-105b46489440 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:06:41.934507 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9486c8a-21a4-4315-91c4-105b46489440 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:06:41.934783 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9486c8a-21a4-4315-91c4-105b46489440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:06:41.936696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c9486c8a-21a4-4315-91c4-105b46489440 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:06:41.996453 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9486c8a-21a4-4315-91c4-105b46489440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:06:41.997895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9486c8a-21a4-4315-91c4-105b46489440 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:06:42.006790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1370/2751] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:06:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:06:42.847991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b12fd2-e8a5-4aa2-acb0-4a496ec7f49b service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:06:42.866093 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b12fd2-e8a5-4aa2-acb0-4a496ec7f49b service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:06:42.872883 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e65bc-5c21-43af-bc2b-6358f826a06d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:06:42.873523 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e65bc-5c21-43af-bc2b-6358f826a06d service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:06:42.873824 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e65bc-5c21-43af-bc2b-6358f826a06d service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:06:42.874084 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e65bc-5c21-43af-bc2b-6358f826a06d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:06:42.876130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-084e65bc-5c21-43af-bc2b-6358f826a06d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:06:42.890966 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e65bc-5c21-43af-bc2b-6358f826a06d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:06:42.892702 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e65bc-5c21-43af-bc2b-6358f826a06d service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:06:42.904319 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1382/2752] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:06:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:06:46.794305 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9486c8a-21a4-4315-91c4-105b46489440 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:06:46.803454 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9486c8a-21a4-4315-91c4-105b46489440 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:06:46.807723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac8bcd-9503-402c-a8db-9fa14a6f7f42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:06:46.808418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac8bcd-9503-402c-a8db-9fa14a6f7f42 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:06:46.808701 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac8bcd-9503-402c-a8db-9fa14a6f7f42 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:06:46.808973 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac8bcd-9503-402c-a8db-9fa14a6f7f42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:06:46.810815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-38ac8bcd-9503-402c-a8db-9fa14a6f7f42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:06:46.874902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac8bcd-9503-402c-a8db-9fa14a6f7f42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:06:46.876361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac8bcd-9503-402c-a8db-9fa14a6f7f42 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:06:46.887056 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1371/2753] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:06:50.404546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e65bc-5c21-43af-bc2b-6358f826a06d service nova] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:06:50.418417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e65bc-5c21-43af-bc2b-6358f826a06d service nova] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:06:50.423468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4168d-5937-4d7f-9365-45c7cd2f8bb6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:06:50.424088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4168d-5937-4d7f-9365-45c7cd2f8bb6 service placement] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:06:50.424412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4168d-5937-4d7f-9365-45c7cd2f8bb6 service placement] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:06:50.424695 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4168d-5937-4d7f-9365-45c7cd2f8bb6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:06:50.426629 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-afc4168d-5937-4d7f-9365-45c7cd2f8bb6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:06:50.439068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4168d-5937-4d7f-9365-45c7cd2f8bb6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:06:50.441078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4168d-5937-4d7f-9365-45c7cd2f8bb6 service placement] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:06:50.454462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1383/2754] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:06:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:06:53.343623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac8bcd-9503-402c-a8db-9fa14a6f7f42 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:06:53.355918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac8bcd-9503-402c-a8db-9fa14a6f7f42 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:06:53.364000 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d577c11-a8a3-4b8d-ab0b-a7e202d1e865 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:06:53.364733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d577c11-a8a3-4b8d-ab0b-a7e202d1e865 service placement] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:06:53.365088 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d577c11-a8a3-4b8d-ab0b-a7e202d1e865 service placement] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:06:53.365433 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d577c11-a8a3-4b8d-ab0b-a7e202d1e865 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:06:53.367333 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9d577c11-a8a3-4b8d-ab0b-a7e202d1e865 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:06:53.381669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d577c11-a8a3-4b8d-ab0b-a7e202d1e865 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:06:53.383540 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d577c11-a8a3-4b8d-ab0b-a7e202d1e865 service placement] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:06:53.406778 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4168d-5937-4d7f-9365-45c7cd2f8bb6 service placement] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:06:53.411089 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1372/2755] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:06:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:06:53.411742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-afc4168d-5937-4d7f-9365-45c7cd2f8bb6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:06:53.427347 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4168d-5937-4d7f-9365-45c7cd2f8bb6 service placement] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:06:53.435355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c98e4-c35e-4a39-bdc3-b6a47e17d827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:06:53.436028 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c98e4-c35e-4a39-bdc3-b6a47e17d827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:06:53.436651 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c98e4-c35e-4a39-bdc3-b6a47e17d827 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:06:53.436845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c98e4-c35e-4a39-bdc3-b6a47e17d827 admin admin] PATH_INFO: `/v3/users/f84fe9ee2c5d48868d73ae651e253f9e` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:06:53.438476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c98e4-c35e-4a39-bdc3-b6a47e17d827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f84fe9ee2c5d48868d73ae651e253f9e)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:06:53.440474 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c98e4-c35e-4a39-bdc3-b6a47e17d827 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:06:53.492565 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-3b4c98e4-c35e-4a39-bdc3-b6a47e17d827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f84fe9ee2c5d48868d73ae651e253f9e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:53.502399 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-3b4c98e4-c35e-4a39-bdc3-b6a47e17d827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f84fe9ee2c5d48868d73ae651e253f9e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:53.517476 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-3b4c98e4-c35e-4a39-bdc3-b6a47e17d827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f84fe9ee2c5d48868d73ae651e253f9e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:53.524268 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-3b4c98e4-c35e-4a39-bdc3-b6a47e17d827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f84fe9ee2c5d48868d73ae651e253f9e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:53.525015 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-3b4c98e4-c35e-4a39-bdc3-b6a47e17d827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f84fe9ee2c5d48868d73ae651e253f9e'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:53.530966 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1384/2756] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:06:53 2022] DELETE /identity/v3/users/f84fe9ee2c5d48868d73ae651e253f9e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:06:53.705264 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d577c11-a8a3-4b8d-ab0b-a7e202d1e865 service placement] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:06:53.716608 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d577c11-a8a3-4b8d-ab0b-a7e202d1e865 service placement] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:06:53.774920 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2a8d7-f672-4625-840d-91b5974384e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:06:53.775544 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2a8d7-f672-4625-840d-91b5974384e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:06:53.776253 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2a8d7-f672-4625-840d-91b5974384e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:06:53.776729 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2a8d7-f672-4625-840d-91b5974384e0 admin admin] PATH_INFO: `/v3/projects/c00e10f7a413422dba1ab74ef86e0946` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:06:53.777994 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2a8d7-f672-4625-840d-91b5974384e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c00e10f7a413422dba1ab74ef86e0946)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:06:53.779904 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2a8d7-f672-4625-840d-91b5974384e0 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:06:53.803168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:06:53.827631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-def2a8d7-f672-4625-840d-91b5974384e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c00e10f7a413422dba1ab74ef86e0946', 'request_id': 'req-def2a8d7-f672-4625-840d-91b5974384e0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:53.832943 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-def2a8d7-f672-4625-840d-91b5974384e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c00e10f7a413422dba1ab74ef86e0946', 'request_id': 'req-def2a8d7-f672-4625-840d-91b5974384e0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:53.839392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-def2a8d7-f672-4625-840d-91b5974384e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c00e10f7a413422dba1ab74ef86e0946', 'request_id': 'req-def2a8d7-f672-4625-840d-91b5974384e0'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:53.852879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-def2a8d7-f672-4625-840d-91b5974384e0 admin admin] The token cache is being invalidate because project c00e10f7a413422dba1ab74ef86e0946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:06:53.853165 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-def2a8d7-f672-4625-840d-91b5974384e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:53.853781 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-def2a8d7-f672-4625-840d-91b5974384e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:53.855855 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1373/2757] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:06:53 2022] DELETE /identity/v3/projects/c00e10f7a413422dba1ab74ef86e0946 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:06:53.859845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c98e4-c35e-4a39-bdc3-b6a47e17d827 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:06:53.861296 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-3b4c98e4-c35e-4a39-bdc3-b6a47e17d827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:06:53.875544 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c98e4-c35e-4a39-bdc3-b6a47e17d827 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:06:53.933302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8dc59-bdca-4008-af62-d93c8d3ecfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:06:53.933993 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8dc59-bdca-4008-af62-d93c8d3ecfea admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:06:53.934281 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8dc59-bdca-4008-af62-d93c8d3ecfea admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:06:53.934593 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8dc59-bdca-4008-af62-d93c8d3ecfea admin admin] PATH_INFO: `/v3/users/7b51e4f2a1664c94bcc57aed111249fd` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:06:53.935809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8dc59-bdca-4008-af62-d93c8d3ecfea admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b51e4f2a1664c94bcc57aed111249fd)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:06:53.937795 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8dc59-bdca-4008-af62-d93c8d3ecfea admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:06:53.986859 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-dac8dc59-bdca-4008-af62-d93c8d3ecfea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b51e4f2a1664c94bcc57aed111249fd'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:53.988457 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:06:53.995231 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-dac8dc59-bdca-4008-af62-d93c8d3ecfea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b51e4f2a1664c94bcc57aed111249fd'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:54.008453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-dac8dc59-bdca-4008-af62-d93c8d3ecfea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b51e4f2a1664c94bcc57aed111249fd'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:54.015718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-dac8dc59-bdca-4008-af62-d93c8d3ecfea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b51e4f2a1664c94bcc57aed111249fd'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:54.016413 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-dac8dc59-bdca-4008-af62-d93c8d3ecfea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b51e4f2a1664c94bcc57aed111249fd'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:54.022366 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1385/2758] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:06:53 2022] DELETE /identity/v3/users/7b51e4f2a1664c94bcc57aed111249fd => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:06:54.202529 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2a8d7-f672-4625-840d-91b5974384e0 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:06:54.204004 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-def2a8d7-f672-4625-840d-91b5974384e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:06:54.218900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2a8d7-f672-4625-840d-91b5974384e0 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:06:54.282532 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7559169-f815-42be-b2c3-49b428ae6254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:06:54.283104 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7559169-f815-42be-b2c3-49b428ae6254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:06:54.283420 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7559169-f815-42be-b2c3-49b428ae6254 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:06:54.283698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7559169-f815-42be-b2c3-49b428ae6254 admin admin] PATH_INFO: `/v3/projects/a16ee75d29f74109aee3b1be4042ac7a` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:06:54.284894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7559169-f815-42be-b2c3-49b428ae6254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a16ee75d29f74109aee3b1be4042ac7a)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:06:54.286835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7559169-f815-42be-b2c3-49b428ae6254 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:06:54.333388 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b7559169-f815-42be-b2c3-49b428ae6254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a16ee75d29f74109aee3b1be4042ac7a', 'request_id': 'req-b7559169-f815-42be-b2c3-49b428ae6254'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:54.339740 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b7559169-f815-42be-b2c3-49b428ae6254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a16ee75d29f74109aee3b1be4042ac7a', 'request_id': 'req-b7559169-f815-42be-b2c3-49b428ae6254'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:54.345997 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b7559169-f815-42be-b2c3-49b428ae6254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a16ee75d29f74109aee3b1be4042ac7a', 'request_id': 'req-b7559169-f815-42be-b2c3-49b428ae6254'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:54.359869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b7559169-f815-42be-b2c3-49b428ae6254 admin admin] The token cache is being invalidate because project a16ee75d29f74109aee3b1be4042ac7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:06:54.360160 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b7559169-f815-42be-b2c3-49b428ae6254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:54.360820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b7559169-f815-42be-b2c3-49b428ae6254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:06:54.362494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1374/2759] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:06:54 2022] DELETE /identity/v3/projects/a16ee75d29f74109aee3b1be4042ac7a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:07:04.018984 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=12 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:07:04.356821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:07:17.408521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befda6c-a7f1-4997-b569-c646d00d5ee7 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:07:17.409040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befda6c-a7f1-4997-b569-c646d00d5ee7 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:07:17.409156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befda6c-a7f1-4997-b569-c646d00d5ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:07:17.429199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-0befda6c-a7f1-4997-b569-c646d00d5ee7 None None] MFA Rules not processed for user `4f7e62d82d054b98bd22dc6958577a74`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:07:17.430410 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0befda6c-a7f1-4997-b569-c646d00d5ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:07:17.484986 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1386/2760] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:07:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:07:18.265041 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7559169-f815-42be-b2c3-49b428ae6254 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:07:18.266741 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b7559169-f815-42be-b2c3-49b428ae6254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:07:18.338031 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7559169-f815-42be-b2c3-49b428ae6254 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:07:18.342934 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05051e4-a1ad-4da0-b4e3-150ecbe6f4e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:07:18.343556 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05051e4-a1ad-4da0-b4e3-150ecbe6f4e2 service placement] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:07:18.343865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05051e4-a1ad-4da0-b4e3-150ecbe6f4e2 service placement] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:07:18.344138 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05051e4-a1ad-4da0-b4e3-150ecbe6f4e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:07:18.345913 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c05051e4-a1ad-4da0-b4e3-150ecbe6f4e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:07:18.357996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05051e4-a1ad-4da0-b4e3-150ecbe6f4e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:07:18.359568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05051e4-a1ad-4da0-b4e3-150ecbe6f4e2 service placement] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:07:18.370105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1375/2761] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:07:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:07:27.477657 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=25 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:07:27.733508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befda6c-a7f1-4997-b569-c646d00d5ee7 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:07:27.734845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-0befda6c-a7f1-4997-b569-c646d00d5ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:07:27.808993 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befda6c-a7f1-4997-b569-c646d00d5ee7 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:07:27.814519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382e22b-a4a3-431d-846a-aafe41685189 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:07:27.815144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382e22b-a4a3-431d-846a-aafe41685189 service nova] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:07:27.815430 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382e22b-a4a3-431d-846a-aafe41685189 service nova] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:07:27.815710 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382e22b-a4a3-431d-846a-aafe41685189 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:07:27.817482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7382e22b-a4a3-431d-846a-aafe41685189 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:07:27.877286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382e22b-a4a3-431d-846a-aafe41685189 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:07:27.878887 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382e22b-a4a3-431d-846a-aafe41685189 service nova] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:07:27.888371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1387/2762] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:07:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:07:28.365375 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=32 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:07:32.314213 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05051e4-a1ad-4da0-b4e3-150ecbe6f4e2 service placement] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:07:32.332611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05051e4-a1ad-4da0-b4e3-150ecbe6f4e2 service placement] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:07:32.340440 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5106798-228b-44a1-ad3c-bc8f7c57106f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:07:32.341099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5106798-228b-44a1-ad3c-bc8f7c57106f service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:07:32.341387 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5106798-228b-44a1-ad3c-bc8f7c57106f service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:07:32.346439 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5106798-228b-44a1-ad3c-bc8f7c57106f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:07:32.349048 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d5106798-228b-44a1-ad3c-bc8f7c57106f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:07:32.376247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5106798-228b-44a1-ad3c-bc8f7c57106f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:07:32.377927 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5106798-228b-44a1-ad3c-bc8f7c57106f service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:07:32.390313 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1376/2763] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:07:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:07:37.883738 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=56 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:07:42.384517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=6 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:07:45.610539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c7fdb-ca4d-4f72-a86f-924bb0349610 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:07:45.611103 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c7fdb-ca4d-4f72-a86f-924bb0349610 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:07:45.611220 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c7fdb-ca4d-4f72-a86f-924bb0349610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:07:45.631542 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-5e2c7fdb-ca4d-4f72-a86f-924bb0349610 None None] MFA Rules not processed for user `cd2bc7050bb9413c8fddd9c7864f537a`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:07:45.632976 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5e2c7fdb-ca4d-4f72-a86f-924bb0349610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:07:45.640765 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1388/2764] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:07:55.626211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:27.403910 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5106798-228b-44a1-ad3c-bc8f7c57106f service nova] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:27.405377 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d5106798-228b-44a1-ad3c-bc8f7c57106f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:27.495172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5106798-228b-44a1-ad3c-bc8f7c57106f service nova] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:27.500621 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801018af-f0b9-45df-b325-65c97af7a937 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:27.501243 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801018af-f0b9-45df-b325-65c97af7a937 service placement] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:27.501523 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801018af-f0b9-45df-b325-65c97af7a937 service placement] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:27.501794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801018af-f0b9-45df-b325-65c97af7a937 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:27.503567 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-801018af-f0b9-45df-b325-65c97af7a937 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:27.516657 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801018af-f0b9-45df-b325-65c97af7a937 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:27.518246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801018af-f0b9-45df-b325-65c97af7a937 service placement] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:27.529153 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1377/2765] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:09:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:09:33.097991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c7fdb-ca4d-4f72-a86f-924bb0349610 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:33.099415 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5e2c7fdb-ca4d-4f72-a86f-924bb0349610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:33.190173 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c7fdb-ca4d-4f72-a86f-924bb0349610 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:33.193707 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f65c2-97ef-4472-8fdb-73223bda8d3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:33.194464 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f65c2-97ef-4472-8fdb-73223bda8d3f service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:33.194741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f65c2-97ef-4472-8fdb-73223bda8d3f service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:33.195529 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f65c2-97ef-4472-8fdb-73223bda8d3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:33.197797 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-cc9f65c2-97ef-4472-8fdb-73223bda8d3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:33.284198 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f65c2-97ef-4472-8fdb-73223bda8d3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:33.285656 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f65c2-97ef-4472-8fdb-73223bda8d3f service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:33.308171 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1389/2766] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:09:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:09:36.358958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801018af-f0b9-45df-b325-65c97af7a937 service placement] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:36.362109 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-801018af-f0b9-45df-b325-65c97af7a937 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:36.487908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801018af-f0b9-45df-b325-65c97af7a937 service placement] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:36.491560 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbbb89-9480-4a2a-a62b-1d7294fbec44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:36.492488 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbbb89-9480-4a2a-a62b-1d7294fbec44 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:36.492801 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbbb89-9480-4a2a-a62b-1d7294fbec44 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:36.493127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbbb89-9480-4a2a-a62b-1d7294fbec44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:36.494979 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-16cbbb89-9480-4a2a-a62b-1d7294fbec44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:36.507222 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbbb89-9480-4a2a-a62b-1d7294fbec44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:36.508821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbbb89-9480-4a2a-a62b-1d7294fbec44 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:36.519166 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1378/2767] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:09:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:09:36.649285 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f65c2-97ef-4472-8fdb-73223bda8d3f service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:36.650782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-cc9f65c2-97ef-4472-8fdb-73223bda8d3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:36.663553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f65c2-97ef-4472-8fdb-73223bda8d3f service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:36.669213 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b5205-f88e-44ae-aecf-7f7cdfa8910b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:36.669871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b5205-f88e-44ae-aecf-7f7cdfa8910b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:36.670157 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b5205-f88e-44ae-aecf-7f7cdfa8910b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:36.670435 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b5205-f88e-44ae-aecf-7f7cdfa8910b admin admin] PATH_INFO: `/v3/users/cd2bc7050bb9413c8fddd9c7864f537a` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:36.671643 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b5205-f88e-44ae-aecf-7f7cdfa8910b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd2bc7050bb9413c8fddd9c7864f537a)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:36.673836 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b5205-f88e-44ae-aecf-7f7cdfa8910b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:36.732535 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:36.736416 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-785b5205-f88e-44ae-aecf-7f7cdfa8910b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd2bc7050bb9413c8fddd9c7864f537a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:36.751000 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-785b5205-f88e-44ae-aecf-7f7cdfa8910b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd2bc7050bb9413c8fddd9c7864f537a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:36.766786 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-785b5205-f88e-44ae-aecf-7f7cdfa8910b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd2bc7050bb9413c8fddd9c7864f537a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:36.776554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-785b5205-f88e-44ae-aecf-7f7cdfa8910b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd2bc7050bb9413c8fddd9c7864f537a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:36.777443 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-785b5205-f88e-44ae-aecf-7f7cdfa8910b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd2bc7050bb9413c8fddd9c7864f537a'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:36.783687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1390/2768] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:09:36 2022] DELETE /identity/v3/users/cd2bc7050bb9413c8fddd9c7864f537a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:09:36.973821 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbbb89-9480-4a2a-a62b-1d7294fbec44 service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:36.997121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbbb89-9480-4a2a-a62b-1d7294fbec44 service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:37.056517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51056e-465d-485d-8119-011354ad9d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:37.057172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51056e-465d-485d-8119-011354ad9d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:37.057452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51056e-465d-485d-8119-011354ad9d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:37.057727 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51056e-465d-485d-8119-011354ad9d7c admin admin] PATH_INFO: `/v3/projects/6a6eee7038a04ad988d383473d0113a4` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:37.058903 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51056e-465d-485d-8119-011354ad9d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a6eee7038a04ad988d383473d0113a4)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:37.060825 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51056e-465d-485d-8119-011354ad9d7c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:37.063082 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:37.109301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-dd51056e-465d-485d-8119-011354ad9d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a6eee7038a04ad988d383473d0113a4', 'request_id': 'req-dd51056e-465d-485d-8119-011354ad9d7c'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:37.114662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-dd51056e-465d-485d-8119-011354ad9d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a6eee7038a04ad988d383473d0113a4', 'request_id': 'req-dd51056e-465d-485d-8119-011354ad9d7c'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:37.121107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-dd51056e-465d-485d-8119-011354ad9d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a6eee7038a04ad988d383473d0113a4', 'request_id': 'req-dd51056e-465d-485d-8119-011354ad9d7c'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:37.133501 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-dd51056e-465d-485d-8119-011354ad9d7c admin admin] The token cache is being invalidate because project 6a6eee7038a04ad988d383473d0113a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:09:37.133796 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-dd51056e-465d-485d-8119-011354ad9d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:37.134418 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-dd51056e-465d-485d-8119-011354ad9d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:37.136178 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1379/2769] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:09:36 2022] DELETE /identity/v3/projects/6a6eee7038a04ad988d383473d0113a4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:09:37.141149 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd48d5-6835-437c-b5b0-fcda62bfac15 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:37.141441 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd48d5-6835-437c-b5b0-fcda62bfac15 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:37.141717 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd48d5-6835-437c-b5b0-fcda62bfac15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:37.201465 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-debd48d5-6835-437c-b5b0-fcda62bfac15 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:09:37.202667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-debd48d5-6835-437c-b5b0-fcda62bfac15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:37.264207 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1391/2770] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 19:09:37.272102 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb3b53-0b88-47ce-9b59-0d580b2411ce None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:37.272426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb3b53-0b88-47ce-9b59-0d580b2411ce None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:37.272710 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb3b53-0b88-47ce-9b59-0d580b2411ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:37.293723 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-49cb3b53-0b88-47ce-9b59-0d580b2411ce None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:09:37.294936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-49cb3b53-0b88-47ce-9b59-0d580b2411ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:37.302958 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1380/2771] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:09:37.307515 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd48d5-6835-437c-b5b0-fcda62bfac15 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:37.317145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd48d5-6835-437c-b5b0-fcda62bfac15 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:37.322239 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1396295-c40d-47bf-a90b-0521b73bb5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:37.322851 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1396295-c40d-47bf-a90b-0521b73bb5fc admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:37.323137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1396295-c40d-47bf-a90b-0521b73bb5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:37.323412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1396295-c40d-47bf-a90b-0521b73bb5fc admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:37.324112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1396295-c40d-47bf-a90b-0521b73bb5fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:09:37.324452 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1396295-c40d-47bf-a90b-0521b73bb5fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:37.326061 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1396295-c40d-47bf-a90b-0521b73bb5fc admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:37.339084 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1392/2772] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:09:37.343413 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb3b53-0b88-47ce-9b59-0d580b2411ce None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:37.353332 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb3b53-0b88-47ce-9b59-0d580b2411ce None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:37.358103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa371c4-1a6f-4cf5-b38c-b33714aa5e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:37.358714 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa371c4-1a6f-4cf5-b38c-b33714aa5e2c admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:37.358999 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa371c4-1a6f-4cf5-b38c-b33714aa5e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:37.359543 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa371c4-1a6f-4cf5-b38c-b33714aa5e2c admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:37.359946 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa371c4-1a6f-4cf5-b38c-b33714aa5e2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:37.361963 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa371c4-1a6f-4cf5-b38c-b33714aa5e2c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:37.389332 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1381/2773] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:09:37 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 19:09:37.393742 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1396295-c40d-47bf-a90b-0521b73bb5fc admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:37.404312 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1396295-c40d-47bf-a90b-0521b73bb5fc admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:37.461950 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9652f13-0b5b-4278-b2d9-ec26a3e08712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:37.462571 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9652f13-0b5b-4278-b2d9-ec26a3e08712 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:37.462852 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9652f13-0b5b-4278-b2d9-ec26a3e08712 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:37.463136 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9652f13-0b5b-4278-b2d9-ec26a3e08712 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:37.463812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9652f13-0b5b-4278-b2d9-ec26a3e08712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:37.465762 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9652f13-0b5b-4278-b2d9-ec26a3e08712 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:37.486831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1393/2774] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:09:37 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:09:37.491148 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa371c4-1a6f-4cf5-b38c-b33714aa5e2c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:37.501580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa371c4-1a6f-4cf5-b38c-b33714aa5e2c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:37.506224 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f0027-2431-449a-a775-3d573e06f69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:37.506838 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f0027-2431-449a-a775-3d573e06f69c admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:37.507127 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f0027-2431-449a-a775-3d573e06f69c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:37.507406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f0027-2431-449a-a775-3d573e06f69c admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:37.509123 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f0027-2431-449a-a775-3d573e06f69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:37.511097 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f0027-2431-449a-a775-3d573e06f69c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:37.521230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1382/2775] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:09:37.526064 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9652f13-0b5b-4278-b2d9-ec26a3e08712 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:37.536400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9652f13-0b5b-4278-b2d9-ec26a3e08712 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:37.541205 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32f98b-3064-436a-b6d4-1109e3d730d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:37.541815 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32f98b-3064-436a-b6d4-1109e3d730d7 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:37.542099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32f98b-3064-436a-b6d4-1109e3d730d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:37.542373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32f98b-3064-436a-b6d4-1109e3d730d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:37.543065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32f98b-3064-436a-b6d4-1109e3d730d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:37.544705 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32f98b-3064-436a-b6d4-1109e3d730d7 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:37.545953 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:37.557485 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1394/2776] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:09:37.560626 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f0027-2431-449a-a775-3d573e06f69c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:37.571124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f0027-2431-449a-a775-3d573e06f69c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:37.576647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff056704-8020-4373-b73f-8ae65fded625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:37.577431 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff056704-8020-4373-b73f-8ae65fded625 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:37.577814 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff056704-8020-4373-b73f-8ae65fded625 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:37.578254 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff056704-8020-4373-b73f-8ae65fded625 admin admin] PATH_INFO: `/v3/projects/7fbdbf49ea1541648ed6d22afdabf891/users/a23b8f68e3094dac80d03274158dec15/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:37.594286 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff056704-8020-4373-b73f-8ae65fded625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fbdbf49ea1541648ed6d22afdabf891, user_id=a23b8f68e3094dac80d03274158dec15, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:37.597018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff056704-8020-4373-b73f-8ae65fded625 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:37.610957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1383/2777] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:09:37 2022] PUT /identity/v3/projects/7fbdbf49ea1541648ed6d22afdabf891/users/a23b8f68e3094dac80d03274158dec15/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:09:37.621990 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd51f96-a75d-45cd-a91d-61617e4fcce1 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:37.622277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd51f96-a75d-45cd-a91d-61617e4fcce1 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:37.622554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd51f96-a75d-45cd-a91d-61617e4fcce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:37.652255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-9dd51f96-a75d-45cd-a91d-61617e4fcce1 None None] MFA Rules not processed for user `a23b8f68e3094dac80d03274158dec15`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:09:37.653486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-9dd51f96-a75d-45cd-a91d-61617e4fcce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:37.719074 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1395/2778] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:09:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:09:37.732836 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff056704-8020-4373-b73f-8ae65fded625 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:37.734566 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ff056704-8020-4373-b73f-8ae65fded625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:37.833503 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:37.842392 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff056704-8020-4373-b73f-8ae65fded625 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:37.848808 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13471f34-7ac6-4412-a8d8-e6c3301edb5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:37.849428 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13471f34-7ac6-4412-a8d8-e6c3301edb5e service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:37.849712 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13471f34-7ac6-4412-a8d8-e6c3301edb5e service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:37.849991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13471f34-7ac6-4412-a8d8-e6c3301edb5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:37.858395 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13471f34-7ac6-4412-a8d8-e6c3301edb5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:37.859975 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13471f34-7ac6-4412-a8d8-e6c3301edb5e service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:37.869019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1384/2779] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:09:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:09:47.716156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=34 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:47.864804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=6 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:52.843079 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d02f9-e244-46d0-aaf3-8bb9fe6c9d85 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:52.843639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d02f9-e244-46d0-aaf3-8bb9fe6c9d85 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:52.843639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d02f9-e244-46d0-aaf3-8bb9fe6c9d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:52.863043 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-025d02f9-e244-46d0-aaf3-8bb9fe6c9d85 None None] MFA Rules not processed for user `4f7e62d82d054b98bd22dc6958577a74`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:09:52.864257 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-025d02f9-e244-46d0-aaf3-8bb9fe6c9d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:52.924202 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1396/2780] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:09:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:09:53.240935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13471f34-7ac6-4412-a8d8-e6c3301edb5e service nova] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:53.243878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-13471f34-7ac6-4412-a8d8-e6c3301edb5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:53.302204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13471f34-7ac6-4412-a8d8-e6c3301edb5e service nova] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:53.306266 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320bdbf-3243-480d-a062-dc65aeba83c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:53.306895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320bdbf-3243-480d-a062-dc65aeba83c1 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:53.307237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320bdbf-3243-480d-a062-dc65aeba83c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:53.307514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320bdbf-3243-480d-a062-dc65aeba83c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:53.309329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-d320bdbf-3243-480d-a062-dc65aeba83c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:53.320683 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320bdbf-3243-480d-a062-dc65aeba83c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:53.322256 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320bdbf-3243-480d-a062-dc65aeba83c1 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:53.333851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1385/2781] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:09:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:09:54.603785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d02f9-e244-46d0-aaf3-8bb9fe6c9d85 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:54.664003 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d02f9-e244-46d0-aaf3-8bb9fe6c9d85 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:54.668579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2640455-c9b0-41f3-8c4c-1ddc06c68a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:54.669224 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2640455-c9b0-41f3-8c4c-1ddc06c68a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:54.669508 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2640455-c9b0-41f3-8c4c-1ddc06c68a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:54.669790 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2640455-c9b0-41f3-8c4c-1ddc06c68a53 admin admin] PATH_INFO: `/v3/users/a23b8f68e3094dac80d03274158dec15` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:54.670950 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2640455-c9b0-41f3-8c4c-1ddc06c68a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a23b8f68e3094dac80d03274158dec15)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:54.672932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2640455-c9b0-41f3-8c4c-1ddc06c68a53 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:54.719849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e2640455-c9b0-41f3-8c4c-1ddc06c68a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a23b8f68e3094dac80d03274158dec15'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:54.727881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e2640455-c9b0-41f3-8c4c-1ddc06c68a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a23b8f68e3094dac80d03274158dec15'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:54.739940 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e2640455-c9b0-41f3-8c4c-1ddc06c68a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a23b8f68e3094dac80d03274158dec15'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:54.747668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e2640455-c9b0-41f3-8c4c-1ddc06c68a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a23b8f68e3094dac80d03274158dec15'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:54.748364 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-e2640455-c9b0-41f3-8c4c-1ddc06c68a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a23b8f68e3094dac80d03274158dec15'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:54.754358 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1397/2782] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:09:54 2022] DELETE /identity/v3/users/a23b8f68e3094dac80d03274158dec15 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:09:54.758884 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84982a4d-1a8c-42be-a56b-ce9ff5b61872 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:54.759384 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84982a4d-1a8c-42be-a56b-ce9ff5b61872 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:54.759492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84982a4d-1a8c-42be-a56b-ce9ff5b61872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:54.778935 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-84982a4d-1a8c-42be-a56b-ce9ff5b61872 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:09:54.780131 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-84982a4d-1a8c-42be-a56b-ce9ff5b61872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:54.841620 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1386/2783] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 19:09:54.851848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2640455-c9b0-41f3-8c4c-1ddc06c68a53 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:54.853910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e2640455-c9b0-41f3-8c4c-1ddc06c68a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:54.890098 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:54.925804 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2640455-c9b0-41f3-8c4c-1ddc06c68a53 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:54.929894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51660692-4f89-41ff-a472-af72736b9417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:54.930861 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51660692-4f89-41ff-a472-af72736b9417 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:54.931299 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51660692-4f89-41ff-a472-af72736b9417 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:54.931715 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51660692-4f89-41ff-a472-af72736b9417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:54.943112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51660692-4f89-41ff-a472-af72736b9417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:54.945876 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51660692-4f89-41ff-a472-af72736b9417 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:54.956727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1398/2784] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:09:55.228100 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84982a4d-1a8c-42be-a56b-ce9ff5b61872 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:55.229729 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-84982a4d-1a8c-42be-a56b-ce9ff5b61872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:55.244587 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84982a4d-1a8c-42be-a56b-ce9ff5b61872 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:55.249447 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80a6a1-f684-448d-8aca-7963dd09be1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:55.250122 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80a6a1-f684-448d-8aca-7963dd09be1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:55.250404 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80a6a1-f684-448d-8aca-7963dd09be1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:55.250682 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80a6a1-f684-448d-8aca-7963dd09be1b admin admin] PATH_INFO: `/v3/projects/7fbdbf49ea1541648ed6d22afdabf891` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:55.251851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80a6a1-f684-448d-8aca-7963dd09be1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fbdbf49ea1541648ed6d22afdabf891)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:55.253813 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80a6a1-f684-448d-8aca-7963dd09be1b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:55.305781 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4d80a6a1-f684-448d-8aca-7963dd09be1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fbdbf49ea1541648ed6d22afdabf891', 'request_id': 'req-4d80a6a1-f684-448d-8aca-7963dd09be1b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:55.311458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4d80a6a1-f684-448d-8aca-7963dd09be1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fbdbf49ea1541648ed6d22afdabf891', 'request_id': 'req-4d80a6a1-f684-448d-8aca-7963dd09be1b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:55.317769 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4d80a6a1-f684-448d-8aca-7963dd09be1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fbdbf49ea1541648ed6d22afdabf891', 'request_id': 'req-4d80a6a1-f684-448d-8aca-7963dd09be1b'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:55.330733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4d80a6a1-f684-448d-8aca-7963dd09be1b admin admin] The token cache is being invalidate because project 7fbdbf49ea1541648ed6d22afdabf891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:09:55.331018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4d80a6a1-f684-448d-8aca-7963dd09be1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:55.331650 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-4d80a6a1-f684-448d-8aca-7963dd09be1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:55.333232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1387/2785] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:09:55 2022] DELETE /identity/v3/projects/7fbdbf49ea1541648ed6d22afdabf891 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 19:09:55.338929 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eabb1c-9b84-4212-9a44-8034faf4d79f None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:55.339726 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eabb1c-9b84-4212-9a44-8034faf4d79f None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:55.339726 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eabb1c-9b84-4212-9a44-8034faf4d79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:55.360585 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-00eabb1c-9b84-4212-9a44-8034faf4d79f None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:09:55.361784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-00eabb1c-9b84-4212-9a44-8034faf4d79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:55.421714 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1399/2786] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:09:55.429843 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647077a-2fa3-4521-b214-c1cdb03e8dc3 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:55.430133 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647077a-2fa3-4521-b214-c1cdb03e8dc3 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:55.430415 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647077a-2fa3-4521-b214-c1cdb03e8dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:55.451482 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-b647077a-2fa3-4521-b214-c1cdb03e8dc3 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:09:55.452744 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b647077a-2fa3-4521-b214-c1cdb03e8dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:55.461202 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1388/2787] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:09:55.465472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eabb1c-9b84-4212-9a44-8034faf4d79f None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:55.475591 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eabb1c-9b84-4212-9a44-8034faf4d79f None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:55.482375 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cec14c-97e1-462c-8433-aece19f190e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:55.482998 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cec14c-97e1-462c-8433-aece19f190e3 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:55.483277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cec14c-97e1-462c-8433-aece19f190e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:55.483544 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cec14c-97e1-462c-8433-aece19f190e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:55.484239 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cec14c-97e1-462c-8433-aece19f190e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:09:55.484570 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cec14c-97e1-462c-8433-aece19f190e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:55.486159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cec14c-97e1-462c-8433-aece19f190e3 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:55.498736 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1400/2788] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:09:55.503184 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647077a-2fa3-4521-b214-c1cdb03e8dc3 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:55.513620 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647077a-2fa3-4521-b214-c1cdb03e8dc3 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:55.518338 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f8d5f-f5dc-45b7-a0ce-413265b9c17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:55.518947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f8d5f-f5dc-45b7-a0ce-413265b9c17a admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:55.519232 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f8d5f-f5dc-45b7-a0ce-413265b9c17a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:55.519509 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f8d5f-f5dc-45b7-a0ce-413265b9c17a admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:55.520172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f8d5f-f5dc-45b7-a0ce-413265b9c17a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:55.522123 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f8d5f-f5dc-45b7-a0ce-413265b9c17a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:55.543259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:55.552021 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1389/2789] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:09:55 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:09:55.556112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cec14c-97e1-462c-8433-aece19f190e3 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:55.566190 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cec14c-97e1-462c-8433-aece19f190e3 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:55.625868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e0ca4-41c3-465c-b5ec-293b02190c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:55.626542 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e0ca4-41c3-465c-b5ec-293b02190c4a admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:55.626759 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e0ca4-41c3-465c-b5ec-293b02190c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:55.627037 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e0ca4-41c3-465c-b5ec-293b02190c4a admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:55.627711 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e0ca4-41c3-465c-b5ec-293b02190c4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:55.629671 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e0ca4-41c3-465c-b5ec-293b02190c4a admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:55.652234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1401/2790] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:09:55 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:09:55.656491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f8d5f-f5dc-45b7-a0ce-413265b9c17a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:55.667696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f8d5f-f5dc-45b7-a0ce-413265b9c17a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:55.672319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf604b-1f92-42be-8e6f-f0ecf035b792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:55.672942 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf604b-1f92-42be-8e6f-f0ecf035b792 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:55.673226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf604b-1f92-42be-8e6f-f0ecf035b792 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:55.673503 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf604b-1f92-42be-8e6f-f0ecf035b792 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:55.674200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf604b-1f92-42be-8e6f-f0ecf035b792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:55.675834 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf604b-1f92-42be-8e6f-f0ecf035b792 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:55.686590 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1390/2791] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:09:55.690873 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e0ca4-41c3-465c-b5ec-293b02190c4a admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:55.701675 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e0ca4-41c3-465c-b5ec-293b02190c4a admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:55.706236 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b723f1bc-cc9b-46ef-b495-7fd3aed12394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:55.706845 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723f1bc-cc9b-46ef-b495-7fd3aed12394 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:55.707131 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723f1bc-cc9b-46ef-b495-7fd3aed12394 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:55.707409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723f1bc-cc9b-46ef-b495-7fd3aed12394 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:55.708104 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b723f1bc-cc9b-46ef-b495-7fd3aed12394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:55.709754 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b723f1bc-cc9b-46ef-b495-7fd3aed12394 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:55.721124 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1402/2792] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:09:55.725291 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf604b-1f92-42be-8e6f-f0ecf035b792 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:55.736029 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf604b-1f92-42be-8e6f-f0ecf035b792 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:55.740894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd210c46-8a12-4c03-b8bd-f57df881923a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:55.741707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd210c46-8a12-4c03-b8bd-f57df881923a admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:55.741996 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd210c46-8a12-4c03-b8bd-f57df881923a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:55.742277 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd210c46-8a12-4c03-b8bd-f57df881923a admin admin] PATH_INFO: `/v3/projects/449ad38526314a1e99e561ceab95b210/users/f2b041ffefec43e7b0172df45fc7f441/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:55.756849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd210c46-8a12-4c03-b8bd-f57df881923a admin admin] RBAC: Authorizing `identity:create_grant(project_id=449ad38526314a1e99e561ceab95b210, user_id=f2b041ffefec43e7b0172df45fc7f441, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:55.760074 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd210c46-8a12-4c03-b8bd-f57df881923a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:55.775091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1391/2793] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:09:55 2022] PUT /identity/v3/projects/449ad38526314a1e99e561ceab95b210/users/f2b041ffefec43e7b0172df45fc7f441/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 19:09:55.787189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad671361-db3a-4645-8f5c-334c127d40fa None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:55.787480 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad671361-db3a-4645-8f5c-334c127d40fa None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:55.787760 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad671361-db3a-4645-8f5c-334c127d40fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:55.815170 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-ad671361-db3a-4645-8f5c-334c127d40fa None None] MFA Rules not processed for user `f2b041ffefec43e7b0172df45fc7f441`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:09:55.816331 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ad671361-db3a-4645-8f5c-334c127d40fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:55.837174 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:55.882687 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1403/2794] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:09:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:09:55.888433 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd210c46-8a12-4c03-b8bd-f57df881923a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:55.986737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd210c46-8a12-4c03-b8bd-f57df881923a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:55.991980 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d04733-f5e3-4e4d-9ce1-83ab613e4e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:55.992697 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d04733-f5e3-4e4d-9ce1-83ab613e4e71 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:55.993105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d04733-f5e3-4e4d-9ce1-83ab613e4e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:55.993390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d04733-f5e3-4e4d-9ce1-83ab613e4e71 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:55.994115 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d04733-f5e3-4e4d-9ce1-83ab613e4e71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:55.995962 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d04733-f5e3-4e4d-9ce1-83ab613e4e71 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:56.030103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1392/2795] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:09:55 2022] POST /identity/v3/projects => generated 363 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:09:56.034238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad671361-db3a-4645-8f5c-334c127d40fa None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:56.048230 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad671361-db3a-4645-8f5c-334c127d40fa None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:56.114086 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4692bce-db38-4057-b362-918d6ef38030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:56.114627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4692bce-db38-4057-b362-918d6ef38030 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:56.114908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4692bce-db38-4057-b362-918d6ef38030 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:56.115189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4692bce-db38-4057-b362-918d6ef38030 admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:56.115853 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4692bce-db38-4057-b362-918d6ef38030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:56.118472 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4692bce-db38-4057-b362-918d6ef38030 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:56.140280 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1404/2796] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:09:56 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 19:09:56.144208 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d04733-f5e3-4e4d-9ce1-83ab613e4e71 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:56.154610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d04733-f5e3-4e4d-9ce1-83ab613e4e71 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:56.159265 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f86ccb-0861-4c3f-98e3-75d0bd718460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:56.159869 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f86ccb-0861-4c3f-98e3-75d0bd718460 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:56.160149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f86ccb-0861-4c3f-98e3-75d0bd718460 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:56.160459 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f86ccb-0861-4c3f-98e3-75d0bd718460 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:56.161149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f86ccb-0861-4c3f-98e3-75d0bd718460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:56.162832 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f86ccb-0861-4c3f-98e3-75d0bd718460 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:56.173730 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1393/2797] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:09:56.177855 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4692bce-db38-4057-b362-918d6ef38030 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:56.188831 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4692bce-db38-4057-b362-918d6ef38030 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:56.193531 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a74a16-4073-436f-9de4-ef4c3e52b44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:56.194201 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a74a16-4073-436f-9de4-ef4c3e52b44e admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:56.194499 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a74a16-4073-436f-9de4-ef4c3e52b44e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:56.195300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a74a16-4073-436f-9de4-ef4c3e52b44e admin admin] PATH_INFO: `/v3/projects/7a72e3152d824b80ab0f3a2c0421237c/users/b0d97f9b52db46618c7d357660b73fbf/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:56.210216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a74a16-4073-436f-9de4-ef4c3e52b44e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a72e3152d824b80ab0f3a2c0421237c, user_id=b0d97f9b52db46618c7d357660b73fbf, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:56.213683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a74a16-4073-436f-9de4-ef4c3e52b44e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:56.227934 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1405/2798] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:09:56 2022] PUT /identity/v3/projects/7a72e3152d824b80ab0f3a2c0421237c/users/b0d97f9b52db46618c7d357660b73fbf/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:09:56.231947 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f86ccb-0861-4c3f-98e3-75d0bd718460 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:56.243034 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f86ccb-0861-4c3f-98e3-75d0bd718460 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:56.300199 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbdfdc-c35e-4216-a2b9-3d401705c2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:56.300841 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbdfdc-c35e-4216-a2b9-3d401705c2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:56.301121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbdfdc-c35e-4216-a2b9-3d401705c2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:56.301394 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbdfdc-c35e-4216-a2b9-3d401705c2d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:56.302083 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbdfdc-c35e-4216-a2b9-3d401705c2d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:56.303767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbdfdc-c35e-4216-a2b9-3d401705c2d0 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:56.314941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1394/2799] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 19:09:56.319241 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a74a16-4073-436f-9de4-ef4c3e52b44e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:56.329871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a74a16-4073-436f-9de4-ef4c3e52b44e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:56.334497 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9782167-98c9-415e-a6fb-ae0760e89375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:56.335225 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9782167-98c9-415e-a6fb-ae0760e89375 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:56.335607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9782167-98c9-415e-a6fb-ae0760e89375 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:56.335892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9782167-98c9-415e-a6fb-ae0760e89375 admin admin] PATH_INFO: `/v3/projects/7a72e3152d824b80ab0f3a2c0421237c/users/b0d97f9b52db46618c7d357660b73fbf/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:56.337944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9782167-98c9-415e-a6fb-ae0760e89375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a72e3152d824b80ab0f3a2c0421237c, user_id=b0d97f9b52db46618c7d357660b73fbf, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:56.340640 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9782167-98c9-415e-a6fb-ae0760e89375 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:56.351849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1406/2800] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:09:56 2022] PUT /identity/v3/projects/7a72e3152d824b80ab0f3a2c0421237c/users/b0d97f9b52db46618c7d357660b73fbf/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:09:56.362381 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72673ae-bb95-4f59-a4fe-ddffba8773b6 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:56.362678 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72673ae-bb95-4f59-a4fe-ddffba8773b6 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:56.362961 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72673ae-bb95-4f59-a4fe-ddffba8773b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:56.393456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-a72673ae-bb95-4f59-a4fe-ddffba8773b6 None None] MFA Rules not processed for user `b0d97f9b52db46618c7d357660b73fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:09:56.394601 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a72673ae-bb95-4f59-a4fe-ddffba8773b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:56.415305 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:56.464081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1395/2801] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:09:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:09:56.475462 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9782167-98c9-415e-a6fb-ae0760e89375 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:56.478174 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e9782167-98c9-415e-a6fb-ae0760e89375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:56.535743 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9782167-98c9-415e-a6fb-ae0760e89375 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:56.540121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee77f3-29e9-4c61-b23e-170ecf089664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:56.540927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee77f3-29e9-4c61-b23e-170ecf089664 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:56.541323 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee77f3-29e9-4c61-b23e-170ecf089664 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:56.541704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee77f3-29e9-4c61-b23e-170ecf089664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:56.551148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee77f3-29e9-4c61-b23e-170ecf089664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:56.552659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee77f3-29e9-4c61-b23e-170ecf089664 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:56.562769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1407/2802] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:09:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:09:56.994644 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72673ae-bb95-4f59-a4fe-ddffba8773b6 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:57.056553 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72673ae-bb95-4f59-a4fe-ddffba8773b6 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:57.061276 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b65a4-00c7-47c6-bc93-7a731e862acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:57.061917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b65a4-00c7-47c6-bc93-7a731e862acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:57.062194 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b65a4-00c7-47c6-bc93-7a731e862acc admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:57.062462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b65a4-00c7-47c6-bc93-7a731e862acc admin admin] PATH_INFO: `/v3/users/f2b041ffefec43e7b0172df45fc7f441` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:57.064247 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b65a4-00c7-47c6-bc93-7a731e862acc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2b041ffefec43e7b0172df45fc7f441)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:57.066540 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b65a4-00c7-47c6-bc93-7a731e862acc admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:57.122025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-810b65a4-00c7-47c6-bc93-7a731e862acc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2b041ffefec43e7b0172df45fc7f441'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:57.132647 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-810b65a4-00c7-47c6-bc93-7a731e862acc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2b041ffefec43e7b0172df45fc7f441'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:57.154046 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-810b65a4-00c7-47c6-bc93-7a731e862acc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2b041ffefec43e7b0172df45fc7f441'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:57.161375 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-810b65a4-00c7-47c6-bc93-7a731e862acc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2b041ffefec43e7b0172df45fc7f441'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:57.162150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-810b65a4-00c7-47c6-bc93-7a731e862acc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2b041ffefec43e7b0172df45fc7f441'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:57.168272 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1396/2803] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:09:56 2022] DELETE /identity/v3/users/f2b041ffefec43e7b0172df45fc7f441 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:09:57.172769 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbf39d-cb53-4206-85b5-dcd97375c6e6 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:57.173246 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbf39d-cb53-4206-85b5-dcd97375c6e6 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:57.173350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbf39d-cb53-4206-85b5-dcd97375c6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:57.193172 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-2cbbf39d-cb53-4206-85b5-dcd97375c6e6 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:09:57.195156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2cbbf39d-cb53-4206-85b5-dcd97375c6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:57.208115 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:57.260469 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1408/2804] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:09:57.270461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b65a4-00c7-47c6-bc93-7a731e862acc admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:57.271900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-810b65a4-00c7-47c6-bc93-7a731e862acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:57.318016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:57.337581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b65a4-00c7-47c6-bc93-7a731e862acc admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:57.342126 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485002c-20d8-48ef-8828-c476189e4601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:57.342756 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485002c-20d8-48ef-8828-c476189e4601 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:57.343040 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485002c-20d8-48ef-8828-c476189e4601 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:57.343314 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485002c-20d8-48ef-8828-c476189e4601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:57.353837 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485002c-20d8-48ef-8828-c476189e4601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:57.356002 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485002c-20d8-48ef-8828-c476189e4601 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:57.366238 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1397/2805] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:09:57.654405 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbf39d-cb53-4206-85b5-dcd97375c6e6 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:57.655864 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-2cbbf39d-cb53-4206-85b5-dcd97375c6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:57.671123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbf39d-cb53-4206-85b5-dcd97375c6e6 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:57.675809 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0a7d5-61d4-4b52-9a10-f559be4afc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:57.676516 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0a7d5-61d4-4b52-9a10-f559be4afc39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:57.676806 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0a7d5-61d4-4b52-9a10-f559be4afc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:57.677203 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0a7d5-61d4-4b52-9a10-f559be4afc39 admin admin] PATH_INFO: `/v3/projects/449ad38526314a1e99e561ceab95b210` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:57.678543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0a7d5-61d4-4b52-9a10-f559be4afc39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=449ad38526314a1e99e561ceab95b210)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:57.681046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0a7d5-61d4-4b52-9a10-f559be4afc39 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:57.731107 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-afb0a7d5-61d4-4b52-9a10-f559be4afc39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '449ad38526314a1e99e561ceab95b210', 'request_id': 'req-afb0a7d5-61d4-4b52-9a10-f559be4afc39'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:57.736550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-afb0a7d5-61d4-4b52-9a10-f559be4afc39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '449ad38526314a1e99e561ceab95b210', 'request_id': 'req-afb0a7d5-61d4-4b52-9a10-f559be4afc39'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:57.744509 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-afb0a7d5-61d4-4b52-9a10-f559be4afc39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '449ad38526314a1e99e561ceab95b210', 'request_id': 'req-afb0a7d5-61d4-4b52-9a10-f559be4afc39'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:57.756743 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-afb0a7d5-61d4-4b52-9a10-f559be4afc39 admin admin] The token cache is being invalidate because project 449ad38526314a1e99e561ceab95b210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:09:57.757071 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-afb0a7d5-61d4-4b52-9a10-f559be4afc39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:57.757728 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-afb0a7d5-61d4-4b52-9a10-f559be4afc39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:57.759521 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1409/2806] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:09:57 2022] DELETE /identity/v3/projects/449ad38526314a1e99e561ceab95b210 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:09:57.763519 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485002c-20d8-48ef-8828-c476189e4601 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:57.773173 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485002c-20d8-48ef-8828-c476189e4601 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:57.833906 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114f5c7-5f0d-4c26-a526-72b12f915a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:57.834564 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114f5c7-5f0d-4c26-a526-72b12f915a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:57.834851 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114f5c7-5f0d-4c26-a526-72b12f915a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:57.835130 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114f5c7-5f0d-4c26-a526-72b12f915a56 admin admin] PATH_INFO: `/v3/users/b0d97f9b52db46618c7d357660b73fbf` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:57.836602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114f5c7-5f0d-4c26-a526-72b12f915a56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0d97f9b52db46618c7d357660b73fbf)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:57.838900 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114f5c7-5f0d-4c26-a526-72b12f915a56 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:57.891848 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7114f5c7-5f0d-4c26-a526-72b12f915a56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0d97f9b52db46618c7d357660b73fbf'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:57.900592 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7114f5c7-5f0d-4c26-a526-72b12f915a56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0d97f9b52db46618c7d357660b73fbf'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:57.921168 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7114f5c7-5f0d-4c26-a526-72b12f915a56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0d97f9b52db46618c7d357660b73fbf'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:57.928742 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7114f5c7-5f0d-4c26-a526-72b12f915a56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0d97f9b52db46618c7d357660b73fbf'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:57.931124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7114f5c7-5f0d-4c26-a526-72b12f915a56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0d97f9b52db46618c7d357660b73fbf'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:57.938737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1398/2807] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:09:57 2022] DELETE /identity/v3/users/b0d97f9b52db46618c7d357660b73fbf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:09:58.233266 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0a7d5-61d4-4b52-9a10-f559be4afc39 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:58.235310 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-afb0a7d5-61d4-4b52-9a10-f559be4afc39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:58.259034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0a7d5-61d4-4b52-9a10-f559be4afc39 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:58.339360 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbda7d2-63e0-48fa-8f27-af61afe812bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:58.340314 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbda7d2-63e0-48fa-8f27-af61afe812bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:58.340653 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbda7d2-63e0-48fa-8f27-af61afe812bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:58.341176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbda7d2-63e0-48fa-8f27-af61afe812bf admin admin] PATH_INFO: `/v3/projects/7a72e3152d824b80ab0f3a2c0421237c` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:58.343185 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbda7d2-63e0-48fa-8f27-af61afe812bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a72e3152d824b80ab0f3a2c0421237c)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:58.345983 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbda7d2-63e0-48fa-8f27-af61afe812bf admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:58.393496 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:58.399209 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4fbda7d2-63e0-48fa-8f27-af61afe812bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a72e3152d824b80ab0f3a2c0421237c', 'request_id': 'req-4fbda7d2-63e0-48fa-8f27-af61afe812bf'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:58.406664 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4fbda7d2-63e0-48fa-8f27-af61afe812bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a72e3152d824b80ab0f3a2c0421237c', 'request_id': 'req-4fbda7d2-63e0-48fa-8f27-af61afe812bf'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:58.415387 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4fbda7d2-63e0-48fa-8f27-af61afe812bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a72e3152d824b80ab0f3a2c0421237c', 'request_id': 'req-4fbda7d2-63e0-48fa-8f27-af61afe812bf'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:58.427372 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4fbda7d2-63e0-48fa-8f27-af61afe812bf admin admin] The token cache is being invalidate because project 7a72e3152d824b80ab0f3a2c0421237c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:09:58.427660 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4fbda7d2-63e0-48fa-8f27-af61afe812bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:58.428326 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-4fbda7d2-63e0-48fa-8f27-af61afe812bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:09:58.430491 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1410/2808] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:09:58 2022] DELETE /identity/v3/projects/7a72e3152d824b80ab0f3a2c0421237c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:09:58.435786 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abc4b4-05af-4f84-9257-7ea5095898cc None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:58.436386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abc4b4-05af-4f84-9257-7ea5095898cc None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:58.436386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abc4b4-05af-4f84-9257-7ea5095898cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:58.456235 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-c2abc4b4-05af-4f84-9257-7ea5095898cc None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:09:58.457585 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-c2abc4b4-05af-4f84-9257-7ea5095898cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:58.512894 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:58.525513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1399/2809] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:09:58.533760 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1568a2-b278-447f-bbe1-fbe5a29c06f7 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:58.534083 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1568a2-b278-447f-bbe1-fbe5a29c06f7 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:58.534436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1568a2-b278-447f-bbe1-fbe5a29c06f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:58.554975 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-6b1568a2-b278-447f-bbe1-fbe5a29c06f7 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:09:58.556458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-6b1568a2-b278-447f-bbe1-fbe5a29c06f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:58.567607 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1411/2810] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 19:09:58.572419 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abc4b4-05af-4f84-9257-7ea5095898cc None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:58.585304 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abc4b4-05af-4f84-9257-7ea5095898cc None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:58.592426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373efd20-f82f-4f5c-953e-8ce93b09815e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:58.593055 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373efd20-f82f-4f5c-953e-8ce93b09815e admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:58.593337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373efd20-f82f-4f5c-953e-8ce93b09815e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:58.593616 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373efd20-f82f-4f5c-953e-8ce93b09815e admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:58.594326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373efd20-f82f-4f5c-953e-8ce93b09815e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:09:58.594620 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373efd20-f82f-4f5c-953e-8ce93b09815e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:58.596214 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373efd20-f82f-4f5c-953e-8ce93b09815e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:58.612580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1400/2811] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:09:58.613703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1568a2-b278-447f-bbe1-fbe5a29c06f7 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:58.626774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1568a2-b278-447f-bbe1-fbe5a29c06f7 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:58.631929 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cb207-50a5-48a8-b89f-dd84281c8d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:58.633431 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cb207-50a5-48a8-b89f-dd84281c8d34 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:58.633703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cb207-50a5-48a8-b89f-dd84281c8d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:58.633703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cb207-50a5-48a8-b89f-dd84281c8d34 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:58.635198 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cb207-50a5-48a8-b89f-dd84281c8d34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:58.637046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cb207-50a5-48a8-b89f-dd84281c8d34 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:58.679706 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1412/2812] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:09:58 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 19:09:58.681189 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373efd20-f82f-4f5c-953e-8ce93b09815e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:58.704767 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373efd20-f82f-4f5c-953e-8ce93b09815e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:58.778619 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b090f-eb8f-4422-a7f1-da848ff06a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:58.779568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b090f-eb8f-4422-a7f1-da848ff06a9c admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:58.780025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b090f-eb8f-4422-a7f1-da848ff06a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:58.780660 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b090f-eb8f-4422-a7f1-da848ff06a9c admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:58.781532 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b090f-eb8f-4422-a7f1-da848ff06a9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:58.784098 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b090f-eb8f-4422-a7f1-da848ff06a9c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:58.815685 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1401/2813] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:09:58 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:09:58.816373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cb207-50a5-48a8-b89f-dd84281c8d34 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:58.829046 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cb207-50a5-48a8-b89f-dd84281c8d34 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:58.837676 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0653fd7-9bfd-4f6a-9739-bdb3667c487b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:58.838333 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0653fd7-9bfd-4f6a-9739-bdb3667c487b admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:58.838595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0653fd7-9bfd-4f6a-9739-bdb3667c487b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:58.838879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0653fd7-9bfd-4f6a-9739-bdb3667c487b admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:58.840142 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0653fd7-9bfd-4f6a-9739-bdb3667c487b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:58.842587 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0653fd7-9bfd-4f6a-9739-bdb3667c487b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:58.857158 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1413/2814] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:09:58.861258 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b090f-eb8f-4422-a7f1-da848ff06a9c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:58.874172 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b090f-eb8f-4422-a7f1-da848ff06a9c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:58.879298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ee990-b501-4ccc-b027-e664cc746b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:58.879862 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ee990-b501-4ccc-b027-e664cc746b66 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:58.880140 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ee990-b501-4ccc-b027-e664cc746b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:58.880444 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ee990-b501-4ccc-b027-e664cc746b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:58.881200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ee990-b501-4ccc-b027-e664cc746b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:58.884042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ee990-b501-4ccc-b027-e664cc746b66 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:58.895435 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1402/2815] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 19:09:58.899547 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0653fd7-9bfd-4f6a-9739-bdb3667c487b admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:58.910373 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0653fd7-9bfd-4f6a-9739-bdb3667c487b admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:58.916078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08a02c-6cd8-4a37-9eaa-12046baff658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:58.916816 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08a02c-6cd8-4a37-9eaa-12046baff658 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:58.917096 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08a02c-6cd8-4a37-9eaa-12046baff658 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:58.917374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08a02c-6cd8-4a37-9eaa-12046baff658 admin admin] PATH_INFO: `/v3/projects/c845adbd681e4f98a88e3553f1d94109/users/031caf05f7e246eb9083adeec798ca39/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:58.934553 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08a02c-6cd8-4a37-9eaa-12046baff658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c845adbd681e4f98a88e3553f1d94109, user_id=031caf05f7e246eb9083adeec798ca39, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:58.937332 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08a02c-6cd8-4a37-9eaa-12046baff658 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:58.951794 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1414/2816] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:09:58 2022] PUT /identity/v3/projects/c845adbd681e4f98a88e3553f1d94109/users/031caf05f7e246eb9083adeec798ca39/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 19:09:58.963895 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faab0fc-1222-462e-979d-65d9ec025e6e None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:58.964469 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faab0fc-1222-462e-979d-65d9ec025e6e None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:58.964950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faab0fc-1222-462e-979d-65d9ec025e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:59.001974 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-6faab0fc-1222-462e-979d-65d9ec025e6e None None] MFA Rules not processed for user `031caf05f7e246eb9083adeec798ca39`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:09:59.002859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-6faab0fc-1222-462e-979d-65d9ec025e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:59.073846 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1403/2817] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:09:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 19:09:59.078397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08a02c-6cd8-4a37-9eaa-12046baff658 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:59.142595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08a02c-6cd8-4a37-9eaa-12046baff658 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:59.147972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0f26a-b5ac-4c88-b346-c1842a48947f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:59.148832 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0f26a-b5ac-4c88-b346-c1842a48947f admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:59.149116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0f26a-b5ac-4c88-b346-c1842a48947f admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:59.149395 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0f26a-b5ac-4c88-b346-c1842a48947f admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:59.150101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0f26a-b5ac-4c88-b346-c1842a48947f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:59.152075 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0f26a-b5ac-4c88-b346-c1842a48947f admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:59.184260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1415/2818] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:09:59 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:09:59.186059 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faab0fc-1222-462e-979d-65d9ec025e6e None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:59.200076 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faab0fc-1222-462e-979d-65d9ec025e6e None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:59.253669 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:59.262604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da54e18-f677-459c-8c26-049d2946864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:59.263300 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da54e18-f677-459c-8c26-049d2946864c admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:59.263580 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da54e18-f677-459c-8c26-049d2946864c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:59.263849 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da54e18-f677-459c-8c26-049d2946864c admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:59.264561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da54e18-f677-459c-8c26-049d2946864c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:59.266681 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da54e18-f677-459c-8c26-049d2946864c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:59.290120 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1404/2819] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:09:59 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 19:09:59.294246 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0f26a-b5ac-4c88-b346-c1842a48947f admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:59.310543 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0f26a-b5ac-4c88-b346-c1842a48947f admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:59.316239 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92c3eb-2da5-4ac3-b457-6866853545d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:59.316907 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92c3eb-2da5-4ac3-b457-6866853545d4 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:59.317823 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92c3eb-2da5-4ac3-b457-6866853545d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:59.317823 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92c3eb-2da5-4ac3-b457-6866853545d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:59.318165 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92c3eb-2da5-4ac3-b457-6866853545d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:59.320216 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92c3eb-2da5-4ac3-b457-6866853545d4 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:59.346108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1416/2820] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:09:59.347648 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da54e18-f677-459c-8c26-049d2946864c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:59.358879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da54e18-f677-459c-8c26-049d2946864c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:59.364085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6bee69-3df0-40de-9eae-d4ccffb9b00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:59.364763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bee69-3df0-40de-9eae-d4ccffb9b00b admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:59.365040 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bee69-3df0-40de-9eae-d4ccffb9b00b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:59.365311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bee69-3df0-40de-9eae-d4ccffb9b00b admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:59.366006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6bee69-3df0-40de-9eae-d4ccffb9b00b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:59.369017 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6bee69-3df0-40de-9eae-d4ccffb9b00b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:59.380375 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1405/2821] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:09:59.384999 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92c3eb-2da5-4ac3-b457-6866853545d4 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:59.396020 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92c3eb-2da5-4ac3-b457-6866853545d4 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:59.401735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86199e-3fd3-4cb4-94fc-62e4c9a17a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:59.402483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86199e-3fd3-4cb4-94fc-62e4c9a17a82 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:59.402771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86199e-3fd3-4cb4-94fc-62e4c9a17a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:59.403212 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86199e-3fd3-4cb4-94fc-62e4c9a17a82 admin admin] PATH_INFO: `/v3/projects/a63743872db941c8856dc5d6c66220a3/users/59429b4bdbc346b0bae6d5bccc608dee/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:59.420055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86199e-3fd3-4cb4-94fc-62e4c9a17a82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63743872db941c8856dc5d6c66220a3, user_id=59429b4bdbc346b0bae6d5bccc608dee, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:59.422894 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86199e-3fd3-4cb4-94fc-62e4c9a17a82 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:59.438575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1417/2822] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:09:59 2022] PUT /identity/v3/projects/a63743872db941c8856dc5d6c66220a3/users/59429b4bdbc346b0bae6d5bccc608dee/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 19:09:59.454662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1462b7-f605-4739-9ee0-8f57b1622520 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:59.455193 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1462b7-f605-4739-9ee0-8f57b1622520 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:59.455301 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1462b7-f605-4739-9ee0-8f57b1622520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:59.493886 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-bd1462b7-f605-4739-9ee0-8f57b1622520 None None] MFA Rules not processed for user `59429b4bdbc346b0bae6d5bccc608dee`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:09:59.495132 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-bd1462b7-f605-4739-9ee0-8f57b1622520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:09:59.578904 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1406/2823] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:09:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:09:59.583436 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86199e-3fd3-4cb4-94fc-62e4c9a17a82 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:59.604351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:09:59.654630 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86199e-3fd3-4cb4-94fc-62e4c9a17a82 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:59.659250 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b45869-7936-4083-9177-1f2a1946d3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:59.659869 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b45869-7936-4083-9177-1f2a1946d3fa admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:59.660150 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b45869-7936-4083-9177-1f2a1946d3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:59.660474 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b45869-7936-4083-9177-1f2a1946d3fa admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:59.661142 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b45869-7936-4083-9177-1f2a1946d3fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:59.663177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b45869-7936-4083-9177-1f2a1946d3fa admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:59.696550 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1418/2824] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:09:59 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:09:59.700403 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1462b7-f605-4739-9ee0-8f57b1622520 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:59.711344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1462b7-f605-4739-9ee0-8f57b1622520 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:59.771751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf967f9-8554-4fca-9bf7-e2e00b9fd7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:59.772412 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf967f9-8554-4fca-9bf7-e2e00b9fd7d2 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:59.772772 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf967f9-8554-4fca-9bf7-e2e00b9fd7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:59.773417 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf967f9-8554-4fca-9bf7-e2e00b9fd7d2 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:59.773765 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf967f9-8554-4fca-9bf7-e2e00b9fd7d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:59.775685 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf967f9-8554-4fca-9bf7-e2e00b9fd7d2 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:59.798820 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1407/2825] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:09:59 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:09:59.802735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b45869-7936-4083-9177-1f2a1946d3fa admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:59.813978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b45869-7936-4083-9177-1f2a1946d3fa admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:59.818682 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1ac63-9c6e-49be-b7db-f122e4e7dc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:59.819300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1ac63-9c6e-49be-b7db-f122e4e7dc7a admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:59.820417 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1ac63-9c6e-49be-b7db-f122e4e7dc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:59.820785 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1ac63-9c6e-49be-b7db-f122e4e7dc7a admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:59.821484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1ac63-9c6e-49be-b7db-f122e4e7dc7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:59.823090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1ac63-9c6e-49be-b7db-f122e4e7dc7a admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:59.835277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1419/2826] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:09:59.839456 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf967f9-8554-4fca-9bf7-e2e00b9fd7d2 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:59.853091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf967f9-8554-4fca-9bf7-e2e00b9fd7d2 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:59.858807 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac461559-af5f-4890-b528-d65d8916abe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:59.859484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac461559-af5f-4890-b528-d65d8916abe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:59.859762 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac461559-af5f-4890-b528-d65d8916abe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:59.860036 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac461559-af5f-4890-b528-d65d8916abe3 admin admin] PATH_INFO: `/v3/projects/7e5dc57360d94c4682d66ff4f7c9c0ae/users/ca9002002b904efc8c8ebd81382772d8/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:59.877030 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac461559-af5f-4890-b528-d65d8916abe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e5dc57360d94c4682d66ff4f7c9c0ae, user_id=ca9002002b904efc8c8ebd81382772d8, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:59.879766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac461559-af5f-4890-b528-d65d8916abe3 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:59.894444 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1408/2827] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:09:59 2022] PUT /identity/v3/projects/7e5dc57360d94c4682d66ff4f7c9c0ae/users/ca9002002b904efc8c8ebd81382772d8/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:09:59.898506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1ac63-9c6e-49be-b7db-f122e4e7dc7a admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:09:59.908474 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1ac63-9c6e-49be-b7db-f122e4e7dc7a admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:09:59.977967 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465b95f-9e26-4ef3-83aa-d4086e2faa9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:09:59.978362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465b95f-9e26-4ef3-83aa-d4086e2faa9c admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:09:59.978474 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465b95f-9e26-4ef3-83aa-d4086e2faa9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:09:59.978729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465b95f-9e26-4ef3-83aa-d4086e2faa9c admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:09:59.979433 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465b95f-9e26-4ef3-83aa-d4086e2faa9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:09:59.981091 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465b95f-9e26-4ef3-83aa-d4086e2faa9c admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:09:59.991956 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1420/2828] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:09:59.996514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac461559-af5f-4890-b528-d65d8916abe3 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:00.006792 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac461559-af5f-4890-b528-d65d8916abe3 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:00.012344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f964aa-d267-45ee-b52b-a96349babd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:00.013018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f964aa-d267-45ee-b52b-a96349babd54 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:00.013297 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f964aa-d267-45ee-b52b-a96349babd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:00.013568 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f964aa-d267-45ee-b52b-a96349babd54 admin admin] PATH_INFO: `/v3/projects/7e5dc57360d94c4682d66ff4f7c9c0ae/users/ca9002002b904efc8c8ebd81382772d8/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:00.015498 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f964aa-d267-45ee-b52b-a96349babd54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e5dc57360d94c4682d66ff4f7c9c0ae, user_id=ca9002002b904efc8c8ebd81382772d8, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:00.018179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f964aa-d267-45ee-b52b-a96349babd54 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:00.029179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1409/2829] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:09:59 2022] PUT /identity/v3/projects/7e5dc57360d94c4682d66ff4f7c9c0ae/users/ca9002002b904efc8c8ebd81382772d8/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:00.040234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45286083-60b3-45d6-b0f5-e9fbc4cee3ae None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:00.040555 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45286083-60b3-45d6-b0f5-e9fbc4cee3ae None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:00.040833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45286083-60b3-45d6-b0f5-e9fbc4cee3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:00.070421 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-45286083-60b3-45d6-b0f5-e9fbc4cee3ae None None] MFA Rules not processed for user `ca9002002b904efc8c8ebd81382772d8`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:00.071619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-45286083-60b3-45d6-b0f5-e9fbc4cee3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:00.142013 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1421/2830] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:10:00.152926 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f964aa-d267-45ee-b52b-a96349babd54 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:00.154611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-13f964aa-d267-45ee-b52b-a96349babd54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:00.174443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:00.216103 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f964aa-d267-45ee-b52b-a96349babd54 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:00.219745 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9aaec-4517-4d0e-9f52-db130e95f6cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:00.220390 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9aaec-4517-4d0e-9f52-db130e95f6cf service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:00.220671 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9aaec-4517-4d0e-9f52-db130e95f6cf service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:00.220942 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9aaec-4517-4d0e-9f52-db130e95f6cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:00.230987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9aaec-4517-4d0e-9f52-db130e95f6cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:00.232436 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9aaec-4517-4d0e-9f52-db130e95f6cf service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:00.242829 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1410/2831] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:10:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:10:00.752530 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45286083-60b3-45d6-b0f5-e9fbc4cee3ae None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:00.753986 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-45286083-60b3-45d6-b0f5-e9fbc4cee3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:00.766394 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45286083-60b3-45d6-b0f5-e9fbc4cee3ae None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:00.770036 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9aebf-d00d-443c-ad36-bd5faf41b595 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:00.770679 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9aebf-d00d-443c-ad36-bd5faf41b595 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:00.770963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9aebf-d00d-443c-ad36-bd5faf41b595 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:00.771238 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9aebf-d00d-443c-ad36-bd5faf41b595 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:00.781059 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:00.830195 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9aebf-d00d-443c-ad36-bd5faf41b595 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:00.831668 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9aebf-d00d-443c-ad36-bd5faf41b595 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:00.841189 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1422/2832] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:10:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:10:05.079678 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9aaec-4517-4d0e-9f52-db130e95f6cf service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:05.088674 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9aaec-4517-4d0e-9f52-db130e95f6cf service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:05.092696 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ceae2-0d6b-4924-a495-6de4689868b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:05.093304 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ceae2-0d6b-4924-a495-6de4689868b4 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:05.093584 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ceae2-0d6b-4924-a495-6de4689868b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:05.093890 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ceae2-0d6b-4924-a495-6de4689868b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:05.151806 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ceae2-0d6b-4924-a495-6de4689868b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:05.153192 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ceae2-0d6b-4924-a495-6de4689868b4 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:05.162099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1411/2833] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:10:05 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:10:07.289357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9aebf-d00d-443c-ad36-bd5faf41b595 service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:07.353482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9aebf-d00d-443c-ad36-bd5faf41b595 service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:07.357950 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5b781-5468-464b-a4a5-ffb278e2c710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:07.358585 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5b781-5468-464b-a4a5-ffb278e2c710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:07.358864 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5b781-5468-464b-a4a5-ffb278e2c710 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:07.359140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5b781-5468-464b-a4a5-ffb278e2c710 admin admin] PATH_INFO: `/v3/users/031caf05f7e246eb9083adeec798ca39` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:07.360505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5b781-5468-464b-a4a5-ffb278e2c710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=031caf05f7e246eb9083adeec798ca39)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:07.362461 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5b781-5468-464b-a4a5-ffb278e2c710 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:07.409854 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-3cb5b781-5468-464b-a4a5-ffb278e2c710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '031caf05f7e246eb9083adeec798ca39'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:07.420715 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-3cb5b781-5468-464b-a4a5-ffb278e2c710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '031caf05f7e246eb9083adeec798ca39'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:07.433662 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-3cb5b781-5468-464b-a4a5-ffb278e2c710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '031caf05f7e246eb9083adeec798ca39'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:07.440439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-3cb5b781-5468-464b-a4a5-ffb278e2c710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '031caf05f7e246eb9083adeec798ca39'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:07.441118 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-3cb5b781-5468-464b-a4a5-ffb278e2c710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '031caf05f7e246eb9083adeec798ca39'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:07.447585 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1423/2834] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:10:07 2022] DELETE /identity/v3/users/031caf05f7e246eb9083adeec798ca39 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:07.452091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3dd260-d2b6-4342-9c00-1ee9b155bc8e None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:07.452589 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3dd260-d2b6-4342-9c00-1ee9b155bc8e None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:07.452691 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3dd260-d2b6-4342-9c00-1ee9b155bc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:07.471720 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-5e3dd260-d2b6-4342-9c00-1ee9b155bc8e None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:07.473037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-5e3dd260-d2b6-4342-9c00-1ee9b155bc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:07.536477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1412/2835] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 19:10:07.546835 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5b781-5468-464b-a4a5-ffb278e2c710 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:07.548241 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-3cb5b781-5468-464b-a4a5-ffb278e2c710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:07.584902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:07.610350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5b781-5468-464b-a4a5-ffb278e2c710 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:07.614090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c845e3-2c40-4da5-9754-72b5c3e304ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:07.614717 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c845e3-2c40-4da5-9754-72b5c3e304ab service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:07.614998 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c845e3-2c40-4da5-9754-72b5c3e304ab service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:07.615270 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c845e3-2c40-4da5-9754-72b5c3e304ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:07.625633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c845e3-2c40-4da5-9754-72b5c3e304ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:07.627071 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c845e3-2c40-4da5-9754-72b5c3e304ab service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:07.637892 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1424/2836] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 19:10:07.986217 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3dd260-d2b6-4342-9c00-1ee9b155bc8e None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:07.987782 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-5e3dd260-d2b6-4342-9c00-1ee9b155bc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:08.002907 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3dd260-d2b6-4342-9c00-1ee9b155bc8e None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:08.009578 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:08.010239 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:08.010528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:08.011865 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe admin admin] PATH_INFO: `/v3/projects/c845adbd681e4f98a88e3553f1d94109` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:08.013639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=c845adbd681e4f98a88e3553f1d94109)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:08.015458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:08.063443 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c845adbd681e4f98a88e3553f1d94109', 'request_id': 'req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.065388 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:08.070582 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c845adbd681e4f98a88e3553f1d94109', 'request_id': 'req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.077139 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c845adbd681e4f98a88e3553f1d94109', 'request_id': 'req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.090180 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe admin admin] The token cache is being invalidate because project c845adbd681e4f98a88e3553f1d94109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:10:08.090472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.091105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.092658 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1413/2837] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:10:07 2022] DELETE /identity/v3/projects/c845adbd681e4f98a88e3553f1d94109 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:08.096893 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c845e3-2c40-4da5-9754-72b5c3e304ab service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:08.106574 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c845e3-2c40-4da5-9754-72b5c3e304ab service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:08.165907 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e7bd1-319b-465d-9d25-4fb4205dd0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:08.166558 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e7bd1-319b-465d-9d25-4fb4205dd0cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:08.166844 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e7bd1-319b-465d-9d25-4fb4205dd0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:08.167118 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e7bd1-319b-465d-9d25-4fb4205dd0cb admin admin] PATH_INFO: `/v3/users/59429b4bdbc346b0bae6d5bccc608dee` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:08.168316 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e7bd1-319b-465d-9d25-4fb4205dd0cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=59429b4bdbc346b0bae6d5bccc608dee)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:08.170259 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e7bd1-319b-465d-9d25-4fb4205dd0cb admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:08.217608 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a48e7bd1-319b-465d-9d25-4fb4205dd0cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59429b4bdbc346b0bae6d5bccc608dee'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.224355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a48e7bd1-319b-465d-9d25-4fb4205dd0cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59429b4bdbc346b0bae6d5bccc608dee'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.236812 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a48e7bd1-319b-465d-9d25-4fb4205dd0cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59429b4bdbc346b0bae6d5bccc608dee'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.243645 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a48e7bd1-319b-465d-9d25-4fb4205dd0cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59429b4bdbc346b0bae6d5bccc608dee'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.244298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a48e7bd1-319b-465d-9d25-4fb4205dd0cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59429b4bdbc346b0bae6d5bccc608dee'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.250121 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1425/2838] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:10:08 2022] DELETE /identity/v3/users/59429b4bdbc346b0bae6d5bccc608dee => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 19:10:08.541881 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:08.543355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:08.556565 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f73277-12b2-48bb-ab2e-f1e7b8e99bfe admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:08.617983 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc8c1e-775b-44f7-9e15-d17df4480c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:08.618640 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc8c1e-775b-44f7-9e15-d17df4480c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:08.618924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc8c1e-775b-44f7-9e15-d17df4480c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:08.619198 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc8c1e-775b-44f7-9e15-d17df4480c27 admin admin] PATH_INFO: `/v3/projects/a63743872db941c8856dc5d6c66220a3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:08.620406 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc8c1e-775b-44f7-9e15-d17df4480c27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63743872db941c8856dc5d6c66220a3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:08.622308 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc8c1e-775b-44f7-9e15-d17df4480c27 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:08.669642 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bdfc8c1e-775b-44f7-9e15-d17df4480c27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63743872db941c8856dc5d6c66220a3', 'request_id': 'req-bdfc8c1e-775b-44f7-9e15-d17df4480c27'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.675236 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bdfc8c1e-775b-44f7-9e15-d17df4480c27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63743872db941c8856dc5d6c66220a3', 'request_id': 'req-bdfc8c1e-775b-44f7-9e15-d17df4480c27'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.682227 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bdfc8c1e-775b-44f7-9e15-d17df4480c27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63743872db941c8856dc5d6c66220a3', 'request_id': 'req-bdfc8c1e-775b-44f7-9e15-d17df4480c27'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.696776 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bdfc8c1e-775b-44f7-9e15-d17df4480c27 admin admin] The token cache is being invalidate because project a63743872db941c8856dc5d6c66220a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:10:08.697061 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bdfc8c1e-775b-44f7-9e15-d17df4480c27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.697728 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bdfc8c1e-775b-44f7-9e15-d17df4480c27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.699298 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1414/2839] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:10:08 2022] DELETE /identity/v3/projects/a63743872db941c8856dc5d6c66220a3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:08.703401 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e7bd1-319b-465d-9d25-4fb4205dd0cb admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:08.704840 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a48e7bd1-319b-465d-9d25-4fb4205dd0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:08.717558 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e7bd1-319b-465d-9d25-4fb4205dd0cb admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:08.765597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:08.776159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23876ef3-975a-4404-95c1-b1086b1b9052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:08.776834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23876ef3-975a-4404-95c1-b1086b1b9052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:08.777118 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23876ef3-975a-4404-95c1-b1086b1b9052 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:08.777393 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23876ef3-975a-4404-95c1-b1086b1b9052 admin admin] PATH_INFO: `/v3/users/ca9002002b904efc8c8ebd81382772d8` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:08.778554 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23876ef3-975a-4404-95c1-b1086b1b9052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca9002002b904efc8c8ebd81382772d8)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:08.780575 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23876ef3-975a-4404-95c1-b1086b1b9052 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:08.830102 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-23876ef3-975a-4404-95c1-b1086b1b9052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca9002002b904efc8c8ebd81382772d8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.845321 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-23876ef3-975a-4404-95c1-b1086b1b9052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca9002002b904efc8c8ebd81382772d8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.860829 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-23876ef3-975a-4404-95c1-b1086b1b9052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca9002002b904efc8c8ebd81382772d8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.868963 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-23876ef3-975a-4404-95c1-b1086b1b9052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca9002002b904efc8c8ebd81382772d8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.869694 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-23876ef3-975a-4404-95c1-b1086b1b9052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca9002002b904efc8c8ebd81382772d8'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:08.876159 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1426/2840] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:10:08 2022] DELETE /identity/v3/users/ca9002002b904efc8c8ebd81382772d8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 19:10:09.243698 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc8c1e-775b-44f7-9e15-d17df4480c27 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:09.245363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-bdfc8c1e-775b-44f7-9e15-d17df4480c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:09.259625 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc8c1e-775b-44f7-9e15-d17df4480c27 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:09.317739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90942fe3-03e7-4039-9e48-911f1348dfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:09.318296 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90942fe3-03e7-4039-9e48-911f1348dfb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:09.318581 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90942fe3-03e7-4039-9e48-911f1348dfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:09.318860 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90942fe3-03e7-4039-9e48-911f1348dfb4 admin admin] PATH_INFO: `/v3/projects/7e5dc57360d94c4682d66ff4f7c9c0ae` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:09.320733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90942fe3-03e7-4039-9e48-911f1348dfb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e5dc57360d94c4682d66ff4f7c9c0ae)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:09.322709 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90942fe3-03e7-4039-9e48-911f1348dfb4 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:09.334058 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:09.371400 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-90942fe3-03e7-4039-9e48-911f1348dfb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e5dc57360d94c4682d66ff4f7c9c0ae', 'request_id': 'req-90942fe3-03e7-4039-9e48-911f1348dfb4'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:09.377067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-90942fe3-03e7-4039-9e48-911f1348dfb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e5dc57360d94c4682d66ff4f7c9c0ae', 'request_id': 'req-90942fe3-03e7-4039-9e48-911f1348dfb4'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:09.384010 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-90942fe3-03e7-4039-9e48-911f1348dfb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e5dc57360d94c4682d66ff4f7c9c0ae', 'request_id': 'req-90942fe3-03e7-4039-9e48-911f1348dfb4'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:09.396775 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-90942fe3-03e7-4039-9e48-911f1348dfb4 admin admin] The token cache is being invalidate because project 7e5dc57360d94c4682d66ff4f7c9c0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:10:09.397150 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-90942fe3-03e7-4039-9e48-911f1348dfb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:09.398155 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-90942fe3-03e7-4039-9e48-911f1348dfb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:09.399697 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1415/2841] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:10:09 2022] DELETE /identity/v3/projects/7e5dc57360d94c4682d66ff4f7c9c0ae => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:09.405639 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c8307-0417-4978-b691-763cabc9767b None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:09.405932 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c8307-0417-4978-b691-763cabc9767b None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:09.406211 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c8307-0417-4978-b691-763cabc9767b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:09.430715 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-df5c8307-0417-4978-b691-763cabc9767b None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:09.431975 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-df5c8307-0417-4978-b691-763cabc9767b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:09.493419 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1427/2842] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:10:09.558602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a663e24-6aec-46a2-bae4-1adb720b1725 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:09.559153 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a663e24-6aec-46a2-bae4-1adb720b1725 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:09.559761 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a663e24-6aec-46a2-bae4-1adb720b1725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:09.580423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-9a663e24-6aec-46a2-bae4-1adb720b1725 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:09.581561 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9a663e24-6aec-46a2-bae4-1adb720b1725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:09.590005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1416/2843] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:10:09.594446 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c8307-0417-4978-b691-763cabc9767b None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:09.605555 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c8307-0417-4978-b691-763cabc9767b None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:09.610227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262a984-2450-472d-9885-a8eec10d8e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:09.610846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262a984-2450-472d-9885-a8eec10d8e00 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:09.611127 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262a984-2450-472d-9885-a8eec10d8e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:09.611400 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262a984-2450-472d-9885-a8eec10d8e00 admin admin] PATH_INFO: `/v3/domains` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:09.612102 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262a984-2450-472d-9885-a8eec10d8e00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91895) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:10:09.612503 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262a984-2450-472d-9885-a8eec10d8e00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:09.614105 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262a984-2450-472d-9885-a8eec10d8e00 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:09.627531 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1428/2844] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:10:09.632639 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a663e24-6aec-46a2-bae4-1adb720b1725 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:09.642397 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a663e24-6aec-46a2-bae4-1adb720b1725 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:09.647237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e758f3-cfe8-42ba-a90b-26361aff4495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:09.647990 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e758f3-cfe8-42ba-a90b-26361aff4495 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:09.648275 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e758f3-cfe8-42ba-a90b-26361aff4495 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:09.648711 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e758f3-cfe8-42ba-a90b-26361aff4495 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:09.649384 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e758f3-cfe8-42ba-a90b-26361aff4495 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:09.651300 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e758f3-cfe8-42ba-a90b-26361aff4495 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:09.680107 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1417/2845] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:10:09 2022] POST /identity/v3/projects => generated 399 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:10:09.684680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262a984-2450-472d-9885-a8eec10d8e00 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:09.696624 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262a984-2450-472d-9885-a8eec10d8e00 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:09.754768 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9209cd-771c-4fd5-a2f5-7dafe73e50ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:09.755420 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9209cd-771c-4fd5-a2f5-7dafe73e50ef admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:09.755647 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9209cd-771c-4fd5-a2f5-7dafe73e50ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:09.755920 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9209cd-771c-4fd5-a2f5-7dafe73e50ef admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:09.756620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9209cd-771c-4fd5-a2f5-7dafe73e50ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:09.759082 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9209cd-771c-4fd5-a2f5-7dafe73e50ef admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:09.766995 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:09.786381 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1429/2846] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:10:09 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 19:10:09.787545 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e758f3-cfe8-42ba-a90b-26361aff4495 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:09.798611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e758f3-cfe8-42ba-a90b-26361aff4495 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:09.803498 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f106e3-1ede-409d-902e-115a3884bdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:09.804703 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f106e3-1ede-409d-902e-115a3884bdf5 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:09.805167 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f106e3-1ede-409d-902e-115a3884bdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:09.805447 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f106e3-1ede-409d-902e-115a3884bdf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:09.806149 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f106e3-1ede-409d-902e-115a3884bdf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:09.807739 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f106e3-1ede-409d-902e-115a3884bdf5 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:09.817835 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1418/2847] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 19:10:09.822397 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9209cd-771c-4fd5-a2f5-7dafe73e50ef admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:09.832548 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9209cd-771c-4fd5-a2f5-7dafe73e50ef admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:09.838426 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5f12b-5159-4aa4-834c-36b4be082266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:09.839041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5f12b-5159-4aa4-834c-36b4be082266 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:09.839322 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5f12b-5159-4aa4-834c-36b4be082266 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:09.839597 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5f12b-5159-4aa4-834c-36b4be082266 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:09.840351 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e5f12b-5159-4aa4-834c-36b4be082266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:09.841959 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e5f12b-5159-4aa4-834c-36b4be082266 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:09.856196 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1430/2848] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:10:09.860355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f106e3-1ede-409d-902e-115a3884bdf5 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:09.871230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f106e3-1ede-409d-902e-115a3884bdf5 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:09.876009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dfd69-68d0-4779-8786-5f10ecd01d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:09.876735 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dfd69-68d0-4779-8786-5f10ecd01d85 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:09.877049 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dfd69-68d0-4779-8786-5f10ecd01d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:09.877329 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dfd69-68d0-4779-8786-5f10ecd01d85 admin admin] PATH_INFO: `/v3/projects/0e1cd22c087b4136b19f88ac07a320de/users/b5646ad563384d89909e0837a8713da9/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:09.892343 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dfd69-68d0-4779-8786-5f10ecd01d85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e1cd22c087b4136b19f88ac07a320de, user_id=b5646ad563384d89909e0837a8713da9, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:09.895015 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dfd69-68d0-4779-8786-5f10ecd01d85 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:09.909243 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1419/2849] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:10:09 2022] PUT /identity/v3/projects/0e1cd22c087b4136b19f88ac07a320de/users/b5646ad563384d89909e0837a8713da9/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:09.921034 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91927388-5e65-46ee-bc32-4afd22b38099 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:09.921321 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91927388-5e65-46ee-bc32-4afd22b38099 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:09.921601 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91927388-5e65-46ee-bc32-4afd22b38099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:09.950570 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-91927388-5e65-46ee-bc32-4afd22b38099 None None] MFA Rules not processed for user `b5646ad563384d89909e0837a8713da9`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:09.951820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-91927388-5e65-46ee-bc32-4afd22b38099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:10.016581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1431/2850] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:09 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:10:10.021124 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dfd69-68d0-4779-8786-5f10ecd01d85 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:10.083702 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dfd69-68d0-4779-8786-5f10ecd01d85 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:10.089708 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2378b3-0494-46a5-9175-07141bd14847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:10.090344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2378b3-0494-46a5-9175-07141bd14847 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:10.090631 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2378b3-0494-46a5-9175-07141bd14847 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:10.090910 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2378b3-0494-46a5-9175-07141bd14847 admin admin] PATH_INFO: `/v3/projects` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:10.091598 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2378b3-0494-46a5-9175-07141bd14847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:10.093536 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2378b3-0494-46a5-9175-07141bd14847 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:10.125006 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1420/2851] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:10:10 2022] POST /identity/v3/projects => generated 401 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:10:10.126483 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91927388-5e65-46ee-bc32-4afd22b38099 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:10.138247 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91927388-5e65-46ee-bc32-4afd22b38099 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:10.198181 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87825a1-6b4c-4ff7-8f18-1f10f8058b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:10.198848 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87825a1-6b4c-4ff7-8f18-1f10f8058b5b admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:10.199144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87825a1-6b4c-4ff7-8f18-1f10f8058b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:10.199453 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87825a1-6b4c-4ff7-8f18-1f10f8058b5b admin admin] PATH_INFO: `/v3/users` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:10.200661 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87825a1-6b4c-4ff7-8f18-1f10f8058b5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:10.202985 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87825a1-6b4c-4ff7-8f18-1f10f8058b5b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:10.224825 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1432/2852] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:10:10 2022] POST /identity/v3/users => generated 307 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:10:10.229506 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2378b3-0494-46a5-9175-07141bd14847 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:10.239950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2378b3-0494-46a5-9175-07141bd14847 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:10.246294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cdae0-b55d-4f44-bed6-2ee595fe8091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:10.246944 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cdae0-b55d-4f44-bed6-2ee595fe8091 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:10.247231 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cdae0-b55d-4f44-bed6-2ee595fe8091 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:10.247538 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cdae0-b55d-4f44-bed6-2ee595fe8091 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:10.248246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cdae0-b55d-4f44-bed6-2ee595fe8091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:10.249876 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cdae0-b55d-4f44-bed6-2ee595fe8091 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:10.263957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1421/2853] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:10:10.267935 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87825a1-6b4c-4ff7-8f18-1f10f8058b5b admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:10.278109 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87825a1-6b4c-4ff7-8f18-1f10f8058b5b admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:10.286194 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39e359-fd74-4f63-a291-bb494e8a7bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:10.287784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39e359-fd74-4f63-a291-bb494e8a7bd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:10.289068 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39e359-fd74-4f63-a291-bb494e8a7bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:10.289620 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39e359-fd74-4f63-a291-bb494e8a7bd9 admin admin] PATH_INFO: `/v3/projects/1eda8aae2b9445b8844abb10ecd95780/users/2af6a77c3ad3420ba5c387f0e25e0e7d/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:10.308336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39e359-fd74-4f63-a291-bb494e8a7bd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eda8aae2b9445b8844abb10ecd95780, user_id=2af6a77c3ad3420ba5c387f0e25e0e7d, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:10.311006 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39e359-fd74-4f63-a291-bb494e8a7bd9 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:10.325655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1433/2854] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:10:10 2022] PUT /identity/v3/projects/1eda8aae2b9445b8844abb10ecd95780/users/2af6a77c3ad3420ba5c387f0e25e0e7d/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:10.329611 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cdae0-b55d-4f44-bed6-2ee595fe8091 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:10.340583 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cdae0-b55d-4f44-bed6-2ee595fe8091 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:10.348295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:10.401075 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84629d-ce6a-4082-bc11-c4fbc0989a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:10.402398 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84629d-ce6a-4082-bc11-c4fbc0989a04 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:10.402692 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84629d-ce6a-4082-bc11-c4fbc0989a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:10.402971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84629d-ce6a-4082-bc11-c4fbc0989a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:10.404240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84629d-ce6a-4082-bc11-c4fbc0989a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:10.407774 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84629d-ce6a-4082-bc11-c4fbc0989a04 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:10.419500 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1422/2855] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:10:10.423457 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39e359-fd74-4f63-a291-bb494e8a7bd9 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:10.434581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39e359-fd74-4f63-a291-bb494e8a7bd9 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:10.439240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7334fe-7317-4c04-9f96-0922ad293fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:10.440036 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7334fe-7317-4c04-9f96-0922ad293fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:10.440344 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7334fe-7317-4c04-9f96-0922ad293fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:10.440622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7334fe-7317-4c04-9f96-0922ad293fb6 admin admin] PATH_INFO: `/v3/projects/1eda8aae2b9445b8844abb10ecd95780/users/2af6a77c3ad3420ba5c387f0e25e0e7d/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:10.442568 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7334fe-7317-4c04-9f96-0922ad293fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eda8aae2b9445b8844abb10ecd95780, user_id=2af6a77c3ad3420ba5c387f0e25e0e7d, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:10.445418 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7334fe-7317-4c04-9f96-0922ad293fb6 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:10.455844 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1434/2856] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:10:10 2022] PUT /identity/v3/projects/1eda8aae2b9445b8844abb10ecd95780/users/2af6a77c3ad3420ba5c387f0e25e0e7d/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:10.467635 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1382b-5c45-460b-bed5-34da6b290000 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:10.467924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1382b-5c45-460b-bed5-34da6b290000 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:10.468201 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1382b-5c45-460b-bed5-34da6b290000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:10.496118 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-4bc1382b-5c45-460b-bed5-34da6b290000 None None] MFA Rules not processed for user `2af6a77c3ad3420ba5c387f0e25e0e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:10.497352 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4bc1382b-5c45-460b-bed5-34da6b290000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:10.568541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1423/2857] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:10 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:10:10.580581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7334fe-7317-4c04-9f96-0922ad293fb6 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:10.582306 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-3c7334fe-7317-4c04-9f96-0922ad293fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:10.605590 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:10.642595 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7334fe-7317-4c04-9f96-0922ad293fb6 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:10.646257 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba0361-4879-4a20-8d40-1440eb43f6f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:10.646874 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba0361-4879-4a20-8d40-1440eb43f6f8 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:10.647156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba0361-4879-4a20-8d40-1440eb43f6f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:10.648090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba0361-4879-4a20-8d40-1440eb43f6f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:10.657356 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba0361-4879-4a20-8d40-1440eb43f6f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:10.658792 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba0361-4879-4a20-8d40-1440eb43f6f8 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:10.668429 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1435/2858] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:10:10 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:10:10.747823 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1382b-5c45-460b-bed5-34da6b290000 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:10.758303 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1382b-5c45-460b-bed5-34da6b290000 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:10.762296 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea4988-bc25-4a3d-b035-f31b61336e9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:10.762924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea4988-bc25-4a3d-b035-f31b61336e9c service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:10.763209 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea4988-bc25-4a3d-b035-f31b61336e9c service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:10.763483 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea4988-bc25-4a3d-b035-f31b61336e9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:10.825188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea4988-bc25-4a3d-b035-f31b61336e9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:10.826638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea4988-bc25-4a3d-b035-f31b61336e9c service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:10.837492 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1424/2859] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:10:10 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:10:14.010036 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba0361-4879-4a20-8d40-1440eb43f6f8 service cinder] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:14.011781 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5dba0361-4879-4a20-8d40-1440eb43f6f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:14.072817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba0361-4879-4a20-8d40-1440eb43f6f8 service cinder] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:14.076991 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c69423-8ac3-4b5d-9486-25fd81836733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:14.077622 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c69423-8ac3-4b5d-9486-25fd81836733 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:14.077910 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c69423-8ac3-4b5d-9486-25fd81836733 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:14.078185 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c69423-8ac3-4b5d-9486-25fd81836733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:14.079980 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-e4c69423-8ac3-4b5d-9486-25fd81836733 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:14.146031 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c69423-8ac3-4b5d-9486-25fd81836733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:14.147482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c69423-8ac3-4b5d-9486-25fd81836733 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:14.158261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1436/2860] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:10:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:10:14.954912 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea4988-bc25-4a3d-b035-f31b61336e9c service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:14.990534 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=100 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:15.022959 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea4988-bc25-4a3d-b035-f31b61336e9c service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:15.027566 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c01d0-9067-4ec7-b183-c8b95e1c8753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:15.028203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c01d0-9067-4ec7-b183-c8b95e1c8753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:15.028513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c01d0-9067-4ec7-b183-c8b95e1c8753 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:15.028788 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c01d0-9067-4ec7-b183-c8b95e1c8753 admin admin] PATH_INFO: `/v3/users/b5646ad563384d89909e0837a8713da9` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:15.030203 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c01d0-9067-4ec7-b183-c8b95e1c8753 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5646ad563384d89909e0837a8713da9)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:15.032122 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c01d0-9067-4ec7-b183-c8b95e1c8753 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:15.080778 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9b3c01d0-9067-4ec7-b183-c8b95e1c8753 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5646ad563384d89909e0837a8713da9'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:15.089044 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9b3c01d0-9067-4ec7-b183-c8b95e1c8753 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5646ad563384d89909e0837a8713da9'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:15.104942 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9b3c01d0-9067-4ec7-b183-c8b95e1c8753 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5646ad563384d89909e0837a8713da9'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:15.112552 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9b3c01d0-9067-4ec7-b183-c8b95e1c8753 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5646ad563384d89909e0837a8713da9'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:15.113294 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-9b3c01d0-9067-4ec7-b183-c8b95e1c8753 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5646ad563384d89909e0837a8713da9'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:15.120026 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1425/2861] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:10:14 2022] DELETE /identity/v3/users/b5646ad563384d89909e0837a8713da9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:15.124409 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff8ae3-67de-4ad3-b755-1f9d9171b43a None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:15.124939 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff8ae3-67de-4ad3-b755-1f9d9171b43a None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:15.125055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff8ae3-67de-4ad3-b755-1f9d9171b43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:15.145882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-8eff8ae3-67de-4ad3-b755-1f9d9171b43a None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:15.147101 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8eff8ae3-67de-4ad3-b755-1f9d9171b43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:15.216486 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1437/2862] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:10:15.228606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c01d0-9067-4ec7-b183-c8b95e1c8753 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:15.230077 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-9b3c01d0-9067-4ec7-b183-c8b95e1c8753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:15.289932 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c01d0-9067-4ec7-b183-c8b95e1c8753 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:15.296123 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffbafe-6948-47ef-b08c-953eb9069c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:15.296819 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffbafe-6948-47ef-b08c-953eb9069c6e service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:15.297101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffbafe-6948-47ef-b08c-953eb9069c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:15.297372 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffbafe-6948-47ef-b08c-953eb9069c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:15.307863 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffbafe-6948-47ef-b08c-953eb9069c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:15.309345 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffbafe-6948-47ef-b08c-953eb9069c6e service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:15.319091 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1426/2863] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:10:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 19:10:15.604691 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff8ae3-67de-4ad3-b755-1f9d9171b43a None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:15.606156 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8eff8ae3-67de-4ad3-b755-1f9d9171b43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:15.618727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff8ae3-67de-4ad3-b755-1f9d9171b43a None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:15.623628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:15.624316 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:15.624612 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:15.624961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2 admin admin] PATH_INFO: `/v3/projects/0e1cd22c087b4136b19f88ac07a320de` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:15.626102 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e1cd22c087b4136b19f88ac07a320de)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:15.628362 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:15.637989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=100 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:15.683482 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e1cd22c087b4136b19f88ac07a320de', 'request_id': 'req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:15.690017 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e1cd22c087b4136b19f88ac07a320de', 'request_id': 'req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:15.701560 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e1cd22c087b4136b19f88ac07a320de', 'request_id': 'req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:15.714157 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2 admin admin] The token cache is being invalidate because project 0e1cd22c087b4136b19f88ac07a320de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:10:15.714706 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:15.715384 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:15.717357 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1438/2864] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:10:15 2022] DELETE /identity/v3/projects/0e1cd22c087b4136b19f88ac07a320de => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:15.721005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffbafe-6948-47ef-b08c-953eb9069c6e service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:15.731537 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffbafe-6948-47ef-b08c-953eb9069c6e service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:15.782878 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:15.794414 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0b00a-dbd6-4b94-b762-b9dcf2fc8985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:15.795317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0b00a-dbd6-4b94-b762-b9dcf2fc8985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:15.795614 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0b00a-dbd6-4b94-b762-b9dcf2fc8985 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:15.796132 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0b00a-dbd6-4b94-b762-b9dcf2fc8985 admin admin] PATH_INFO: `/v3/users/2af6a77c3ad3420ba5c387f0e25e0e7d` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:15.797871 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0b00a-dbd6-4b94-b762-b9dcf2fc8985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2af6a77c3ad3420ba5c387f0e25e0e7d)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:15.799896 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0b00a-dbd6-4b94-b762-b9dcf2fc8985 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:15.852399 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ef0b00a-dbd6-4b94-b762-b9dcf2fc8985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2af6a77c3ad3420ba5c387f0e25e0e7d'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:15.860422 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ef0b00a-dbd6-4b94-b762-b9dcf2fc8985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2af6a77c3ad3420ba5c387f0e25e0e7d'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:15.873393 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ef0b00a-dbd6-4b94-b762-b9dcf2fc8985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2af6a77c3ad3420ba5c387f0e25e0e7d'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:15.880383 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ef0b00a-dbd6-4b94-b762-b9dcf2fc8985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2af6a77c3ad3420ba5c387f0e25e0e7d'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:15.881337 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-5ef0b00a-dbd6-4b94-b762-b9dcf2fc8985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2af6a77c3ad3420ba5c387f0e25e0e7d'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:15.888803 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1427/2865] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:10:15 2022] DELETE /identity/v3/users/2af6a77c3ad3420ba5c387f0e25e0e7d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:16.186199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:16.187791 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:16.204322 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765bac2-6a9e-4b8b-85d2-e962b67f16f2 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:16.299568 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72d27b-ef59-4ab0-a865-8a859be2a623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:16.300383 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72d27b-ef59-4ab0-a865-8a859be2a623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:16.300703 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72d27b-ef59-4ab0-a865-8a859be2a623 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:16.301236 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72d27b-ef59-4ab0-a865-8a859be2a623 admin admin] PATH_INFO: `/v3/projects/1eda8aae2b9445b8844abb10ecd95780` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:16.302860 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72d27b-ef59-4ab0-a865-8a859be2a623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eda8aae2b9445b8844abb10ecd95780)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:16.306440 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72d27b-ef59-4ab0-a865-8a859be2a623 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:16.376050 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-aa72d27b-ef59-4ab0-a865-8a859be2a623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eda8aae2b9445b8844abb10ecd95780', 'request_id': 'req-aa72d27b-ef59-4ab0-a865-8a859be2a623'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:16.384650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-aa72d27b-ef59-4ab0-a865-8a859be2a623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eda8aae2b9445b8844abb10ecd95780', 'request_id': 'req-aa72d27b-ef59-4ab0-a865-8a859be2a623'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:16.394548 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-aa72d27b-ef59-4ab0-a865-8a859be2a623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eda8aae2b9445b8844abb10ecd95780', 'request_id': 'req-aa72d27b-ef59-4ab0-a865-8a859be2a623'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:16.409142 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-aa72d27b-ef59-4ab0-a865-8a859be2a623 admin admin] The token cache is being invalidate because project 1eda8aae2b9445b8844abb10ecd95780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:10:16.409439 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-aa72d27b-ef59-4ab0-a865-8a859be2a623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:16.412137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-aa72d27b-ef59-4ab0-a865-8a859be2a623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:16.414116 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1439/2866] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:10:16 2022] DELETE /identity/v3/projects/1eda8aae2b9445b8844abb10ecd95780 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:16.419719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced01e3f-d23e-4e0e-9ea6-4f9fc3388cb2 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:16.419719 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced01e3f-d23e-4e0e-9ea6-4f9fc3388cb2 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:16.420419 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced01e3f-d23e-4e0e-9ea6-4f9fc3388cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:16.441468 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-ced01e3f-d23e-4e0e-9ea6-4f9fc3388cb2 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:16.442688 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-ced01e3f-d23e-4e0e-9ea6-4f9fc3388cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:16.519790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1428/2867] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 19:10:16.528834 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcbb56-f508-48f4-8767-1053c5824f68 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:16.529117 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcbb56-f508-48f4-8767-1053c5824f68 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:16.529391 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcbb56-f508-48f4-8767-1053c5824f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:16.551817 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-02fcbb56-f508-48f4-8767-1053c5824f68 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:16.553177 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-02fcbb56-f508-48f4-8767-1053c5824f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:16.565157 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1440/2868] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 19:10:16.572230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced01e3f-d23e-4e0e-9ea6-4f9fc3388cb2 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:16.582768 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced01e3f-d23e-4e0e-9ea6-4f9fc3388cb2 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:16.588121 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c79ba-6582-4907-8ae6-eb8ec50db463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:16.588790 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c79ba-6582-4907-8ae6-eb8ec50db463 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:16.589070 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c79ba-6582-4907-8ae6-eb8ec50db463 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:16.589342 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c79ba-6582-4907-8ae6-eb8ec50db463 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:16.590042 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c79ba-6582-4907-8ae6-eb8ec50db463 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:10:16.590342 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c79ba-6582-4907-8ae6-eb8ec50db463 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:16.592224 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c79ba-6582-4907-8ae6-eb8ec50db463 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:16.608097 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1429/2869] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:10:16.609258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcbb56-f508-48f4-8767-1053c5824f68 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:16.619727 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcbb56-f508-48f4-8767-1053c5824f68 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:16.624519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646ffac-c895-4acf-a341-fa70addfc6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:16.625144 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646ffac-c895-4acf-a341-fa70addfc6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:16.625458 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646ffac-c895-4acf-a341-fa70addfc6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:16.625735 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646ffac-c895-4acf-a341-fa70addfc6d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:16.626419 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646ffac-c895-4acf-a341-fa70addfc6d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:16.628652 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646ffac-c895-4acf-a341-fa70addfc6d2 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:16.654038 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:16.667526 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1441/2870] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:10:16 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:10:16.668757 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c79ba-6582-4907-8ae6-eb8ec50db463 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:16.678819 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c79ba-6582-4907-8ae6-eb8ec50db463 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:16.752754 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e6727-b0e8-4df3-99fb-17fc8b05306a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:16.753382 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e6727-b0e8-4df3-99fb-17fc8b05306a admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:16.753661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e6727-b0e8-4df3-99fb-17fc8b05306a admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:16.753936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e6727-b0e8-4df3-99fb-17fc8b05306a admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:16.754604 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e6727-b0e8-4df3-99fb-17fc8b05306a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:16.756548 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e6727-b0e8-4df3-99fb-17fc8b05306a admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:16.766067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:16.786909 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1430/2871] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:10:16 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:10:16.788407 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646ffac-c895-4acf-a341-fa70addfc6d2 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:16.802633 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646ffac-c895-4acf-a341-fa70addfc6d2 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:16.809140 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c05c2-7348-41b6-991e-2390a62637dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:16.810070 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c05c2-7348-41b6-991e-2390a62637dc admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:16.810485 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c05c2-7348-41b6-991e-2390a62637dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:16.810923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c05c2-7348-41b6-991e-2390a62637dc admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:16.811638 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c05c2-7348-41b6-991e-2390a62637dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:16.813284 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c05c2-7348-41b6-991e-2390a62637dc admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:16.825931 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1442/2872] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 19:10:16.830393 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e6727-b0e8-4df3-99fb-17fc8b05306a admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:16.840968 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e6727-b0e8-4df3-99fb-17fc8b05306a admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:16.847068 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6e637-aa66-49be-8310-e167b6165e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:16.847763 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6e637-aa66-49be-8310-e167b6165e1e admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:16.848050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6e637-aa66-49be-8310-e167b6165e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:16.848411 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6e637-aa66-49be-8310-e167b6165e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:16.849129 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6e637-aa66-49be-8310-e167b6165e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:16.850815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6e637-aa66-49be-8310-e167b6165e1e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:16.862664 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1431/2873] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 19:10:16.867764 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c05c2-7348-41b6-991e-2390a62637dc admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:16.878193 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c05c2-7348-41b6-991e-2390a62637dc admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:16.883157 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a200d96-87e4-4f0e-834c-7518b7785082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:16.883884 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a200d96-87e4-4f0e-834c-7518b7785082 admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:16.884252 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a200d96-87e4-4f0e-834c-7518b7785082 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:16.884627 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a200d96-87e4-4f0e-834c-7518b7785082 admin admin] PATH_INFO: `/v3/projects/ffd95f8bb3d74989a9c304c9c2e5e437/users/277c30401ffe45df9a168dce3fa77174/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:16.900484 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a200d96-87e4-4f0e-834c-7518b7785082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd95f8bb3d74989a9c304c9c2e5e437, user_id=277c30401ffe45df9a168dce3fa77174, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:16.903152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a200d96-87e4-4f0e-834c-7518b7785082 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:16.924510 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1443/2874] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:10:16 2022] PUT /identity/v3/projects/ffd95f8bb3d74989a9c304c9c2e5e437/users/277c30401ffe45df9a168dce3fa77174/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 19:10:16.936704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc4e9b-42eb-4520-9f28-0a7b697b0aa5 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:16.936704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc4e9b-42eb-4520-9f28-0a7b697b0aa5 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:16.936704 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc4e9b-42eb-4520-9f28-0a7b697b0aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:16.968355 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-0dfc4e9b-42eb-4520-9f28-0a7b697b0aa5 None None] MFA Rules not processed for user `277c30401ffe45df9a168dce3fa77174`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:16.969624 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-0dfc4e9b-42eb-4520-9f28-0a7b697b0aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:17.042426 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1432/2875] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:10:17.047123 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a200d96-87e4-4f0e-834c-7518b7785082 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:17.114650 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a200d96-87e4-4f0e-834c-7518b7785082 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:17.119988 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55818852-7e10-4669-aac2-2c58a70f8e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:17.120659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55818852-7e10-4669-aac2-2c58a70f8e13 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:17.120940 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55818852-7e10-4669-aac2-2c58a70f8e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:17.121241 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55818852-7e10-4669-aac2-2c58a70f8e13 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:17.121923 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55818852-7e10-4669-aac2-2c58a70f8e13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:17.123837 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55818852-7e10-4669-aac2-2c58a70f8e13 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:17.155240 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1444/2876] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:10:17 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:10:17.159326 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc4e9b-42eb-4520-9f28-0a7b697b0aa5 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:17.173913 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc4e9b-42eb-4520-9f28-0a7b697b0aa5 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:17.239452 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f391f-0cc5-406f-be39-df940acd1c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:17.240081 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f391f-0cc5-406f-be39-df940acd1c65 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:17.240403 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f391f-0cc5-406f-be39-df940acd1c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:17.240682 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f391f-0cc5-406f-be39-df940acd1c65 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:17.241357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f391f-0cc5-406f-be39-df940acd1c65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:17.243936 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f391f-0cc5-406f-be39-df940acd1c65 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:17.270245 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1433/2877] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:10:17 2022] POST /identity/v3/users => generated 298 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 19:10:17.271493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55818852-7e10-4669-aac2-2c58a70f8e13 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:17.282383 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55818852-7e10-4669-aac2-2c58a70f8e13 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:17.287137 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41b87d-dfce-4f83-a134-53fb5b65550b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:17.287771 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41b87d-dfce-4f83-a134-53fb5b65550b admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:17.288055 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41b87d-dfce-4f83-a134-53fb5b65550b admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:17.288374 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41b87d-dfce-4f83-a134-53fb5b65550b admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:17.289090 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed41b87d-dfce-4f83-a134-53fb5b65550b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:17.290704 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed41b87d-dfce-4f83-a134-53fb5b65550b admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:17.301748 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1445/2878] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:10:17.306737 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f391f-0cc5-406f-be39-df940acd1c65 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:17.319065 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f391f-0cc5-406f-be39-df940acd1c65 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:17.324917 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b106478-c733-4401-a2c8-a54633cdc62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:17.325706 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b106478-c733-4401-a2c8-a54633cdc62b admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:17.325991 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b106478-c733-4401-a2c8-a54633cdc62b admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:17.326285 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b106478-c733-4401-a2c8-a54633cdc62b admin admin] PATH_INFO: `/v3/projects/db6371b090764f7296f86cd1baa66e28/users/522ad1fff47a406fb97ac7f1d1fc920e/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:17.342956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b106478-c733-4401-a2c8-a54633cdc62b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db6371b090764f7296f86cd1baa66e28, user_id=522ad1fff47a406fb97ac7f1d1fc920e, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:17.345819 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b106478-c733-4401-a2c8-a54633cdc62b admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:17.360059 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1434/2879] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:10:17 2022] PUT /identity/v3/projects/db6371b090764f7296f86cd1baa66e28/users/522ad1fff47a406fb97ac7f1d1fc920e/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 19:10:17.364030 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41b87d-dfce-4f83-a134-53fb5b65550b admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:17.374619 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41b87d-dfce-4f83-a134-53fb5b65550b admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:17.431927 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71509627-5b12-4200-9897-fd6d139a1e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:17.432623 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71509627-5b12-4200-9897-fd6d139a1e79 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:17.432907 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71509627-5b12-4200-9897-fd6d139a1e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:17.433181 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71509627-5b12-4200-9897-fd6d139a1e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:17.433881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71509627-5b12-4200-9897-fd6d139a1e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:17.435541 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71509627-5b12-4200-9897-fd6d139a1e79 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:17.446255 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1446/2880] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:10:17.450484 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b106478-c733-4401-a2c8-a54633cdc62b admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:17.461062 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b106478-c733-4401-a2c8-a54633cdc62b admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:17.465655 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf9ea9-24d5-4f3c-81c3-652471d912e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:17.466317 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf9ea9-24d5-4f3c-81c3-652471d912e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:17.466599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf9ea9-24d5-4f3c-81c3-652471d912e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:17.466872 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf9ea9-24d5-4f3c-81c3-652471d912e7 admin admin] PATH_INFO: `/v3/projects/db6371b090764f7296f86cd1baa66e28/users/522ad1fff47a406fb97ac7f1d1fc920e/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:17.468845 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf9ea9-24d5-4f3c-81c3-652471d912e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db6371b090764f7296f86cd1baa66e28, user_id=522ad1fff47a406fb97ac7f1d1fc920e, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:17.471487 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf9ea9-24d5-4f3c-81c3-652471d912e7 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:17.481956 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1435/2881] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:10:17 2022] PUT /identity/v3/projects/db6371b090764f7296f86cd1baa66e28/users/522ad1fff47a406fb97ac7f1d1fc920e/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:17.493798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ac8f3-045c-4939-8337-01e1ac3f6726 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:17.494021 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ac8f3-045c-4939-8337-01e1ac3f6726 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:17.494303 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ac8f3-045c-4939-8337-01e1ac3f6726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:17.525300 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-5b1ac8f3-045c-4939-8337-01e1ac3f6726 None None] MFA Rules not processed for user `522ad1fff47a406fb97ac7f1d1fc920e`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:17.526487 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-5b1ac8f3-045c-4939-8337-01e1ac3f6726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:17.546467 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:17.596843 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1447/2882] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:10:17.608640 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf9ea9-24d5-4f3c-81c3-652471d912e7 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:17.610316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-bacf9ea9-24d5-4f3c-81c3-652471d912e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:17.635885 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:17.674119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf9ea9-24d5-4f3c-81c3-652471d912e7 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:17.677854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ba94d-c5b1-4262-9119-e44d2feeda4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:17.678491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ba94d-c5b1-4262-9119-e44d2feeda4e service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:17.678782 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ba94d-c5b1-4262-9119-e44d2feeda4e service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:17.679090 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ba94d-c5b1-4262-9119-e44d2feeda4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:17.688791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ba94d-c5b1-4262-9119-e44d2feeda4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:17.690229 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ba94d-c5b1-4262-9119-e44d2feeda4e service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:17.700141 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1436/2883] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:10:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:10:17.802919 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ac8f3-045c-4939-8337-01e1ac3f6726 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:17.813258 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ac8f3-045c-4939-8337-01e1ac3f6726 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:17.816906 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8740e10-7311-4794-b55e-c3d0a138c7d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:17.817519 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8740e10-7311-4794-b55e-c3d0a138c7d0 service cinder] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:17.817802 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8740e10-7311-4794-b55e-c3d0a138c7d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:17.818078 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8740e10-7311-4794-b55e-c3d0a138c7d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:17.877498 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8740e10-7311-4794-b55e-c3d0a138c7d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:17.878966 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8740e10-7311-4794-b55e-c3d0a138c7d0 service cinder] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:17.892882 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1448/2884] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:10:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:10:20.498170 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ba94d-c5b1-4262-9119-e44d2feeda4e service cinder] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:20.561602 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ba94d-c5b1-4262-9119-e44d2feeda4e service cinder] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:20.566573 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b447d-551f-4671-bd9f-5d3a6efb710e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:20.567307 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b447d-551f-4671-bd9f-5d3a6efb710e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:20.567592 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b447d-551f-4671-bd9f-5d3a6efb710e admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:20.567872 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b447d-551f-4671-bd9f-5d3a6efb710e admin admin] PATH_INFO: `/v3/users/277c30401ffe45df9a168dce3fa77174` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:20.569246 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b447d-551f-4671-bd9f-5d3a6efb710e admin admin] RBAC: Authorizing `identity:delete_user(user_id=277c30401ffe45df9a168dce3fa77174)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:20.571287 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b447d-551f-4671-bd9f-5d3a6efb710e admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:20.621551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7b9b447d-551f-4671-bd9f-5d3a6efb710e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '277c30401ffe45df9a168dce3fa77174'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:20.628610 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7b9b447d-551f-4671-bd9f-5d3a6efb710e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '277c30401ffe45df9a168dce3fa77174'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:20.641517 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7b9b447d-551f-4671-bd9f-5d3a6efb710e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '277c30401ffe45df9a168dce3fa77174'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:20.649259 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7b9b447d-551f-4671-bd9f-5d3a6efb710e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '277c30401ffe45df9a168dce3fa77174'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:20.649942 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-7b9b447d-551f-4671-bd9f-5d3a6efb710e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '277c30401ffe45df9a168dce3fa77174'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:20.655971 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1437/2885] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:10:20 2022] DELETE /identity/v3/users/277c30401ffe45df9a168dce3fa77174 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:20.660475 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5f579-e9cb-42c1-8c3a-a40e84858d77 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:20.660972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5f579-e9cb-42c1-8c3a-a40e84858d77 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:20.661081 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5f579-e9cb-42c1-8c3a-a40e84858d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:20.679784 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-a8a5f579-e9cb-42c1-8c3a-a40e84858d77 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:20.681029 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a8a5f579-e9cb-42c1-8c3a-a40e84858d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:20.740239 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1449/2886] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:10:20.750101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b447d-551f-4671-bd9f-5d3a6efb710e admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:20.751495 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-7b9b447d-551f-4671-bd9f-5d3a6efb710e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:20.797773 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:20.811901 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b447d-551f-4671-bd9f-5d3a6efb710e admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:20.815666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a6142-f254-4cb2-95ea-1e6b0a58a6d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:20.816296 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a6142-f254-4cb2-95ea-1e6b0a58a6d8 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:20.816623 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a6142-f254-4cb2-95ea-1e6b0a58a6d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:20.816902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a6142-f254-4cb2-95ea-1e6b0a58a6d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:20.826815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a6142-f254-4cb2-95ea-1e6b0a58a6d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:20.828274 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a6142-f254-4cb2-95ea-1e6b0a58a6d8 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:20.837831 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1438/2887] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:10:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:10:21.111875 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5f579-e9cb-42c1-8c3a-a40e84858d77 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:21.113448 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-a8a5f579-e9cb-42c1-8c3a-a40e84858d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:21.126667 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5f579-e9cb-42c1-8c3a-a40e84858d77 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:21.131539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3923b2-e107-4042-96ed-a24c989320fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:21.132971 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3923b2-e107-4042-96ed-a24c989320fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:21.133854 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3923b2-e107-4042-96ed-a24c989320fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:21.133854 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3923b2-e107-4042-96ed-a24c989320fc admin admin] PATH_INFO: `/v3/projects/ffd95f8bb3d74989a9c304c9c2e5e437` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:21.134851 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3923b2-e107-4042-96ed-a24c989320fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffd95f8bb3d74989a9c304c9c2e5e437)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:21.137969 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3923b2-e107-4042-96ed-a24c989320fc admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:21.188277 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ee3923b2-e107-4042-96ed-a24c989320fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffd95f8bb3d74989a9c304c9c2e5e437', 'request_id': 'req-ee3923b2-e107-4042-96ed-a24c989320fc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:21.190298 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:21.195316 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ee3923b2-e107-4042-96ed-a24c989320fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffd95f8bb3d74989a9c304c9c2e5e437', 'request_id': 'req-ee3923b2-e107-4042-96ed-a24c989320fc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:21.203325 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ee3923b2-e107-4042-96ed-a24c989320fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffd95f8bb3d74989a9c304c9c2e5e437', 'request_id': 'req-ee3923b2-e107-4042-96ed-a24c989320fc'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:21.214312 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ee3923b2-e107-4042-96ed-a24c989320fc admin admin] The token cache is being invalidate because project ffd95f8bb3d74989a9c304c9c2e5e437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:10:21.214638 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ee3923b2-e107-4042-96ed-a24c989320fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:21.215355 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-ee3923b2-e107-4042-96ed-a24c989320fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:21.216918 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1450/2888] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:10:21 2022] DELETE /identity/v3/projects/ffd95f8bb3d74989a9c304c9c2e5e437 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:21.220987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a6142-f254-4cb2-95ea-1e6b0a58a6d8 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:21.231035 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a6142-f254-4cb2-95ea-1e6b0a58a6d8 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:21.289295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0872cf3-727f-4145-abc4-a8822b17daca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:21.289872 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0872cf3-727f-4145-abc4-a8822b17daca admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:21.290159 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0872cf3-727f-4145-abc4-a8822b17daca admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:21.290467 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0872cf3-727f-4145-abc4-a8822b17daca admin admin] PATH_INFO: `/v3/users/522ad1fff47a406fb97ac7f1d1fc920e` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:21.291662 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0872cf3-727f-4145-abc4-a8822b17daca admin admin] RBAC: Authorizing `identity:delete_user(user_id=522ad1fff47a406fb97ac7f1d1fc920e)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:21.293690 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0872cf3-727f-4145-abc4-a8822b17daca admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:21.341815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-d0872cf3-727f-4145-abc4-a8822b17daca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '522ad1fff47a406fb97ac7f1d1fc920e'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:21.349050 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-d0872cf3-727f-4145-abc4-a8822b17daca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '522ad1fff47a406fb97ac7f1d1fc920e'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:21.361357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-d0872cf3-727f-4145-abc4-a8822b17daca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '522ad1fff47a406fb97ac7f1d1fc920e'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:21.368344 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-d0872cf3-727f-4145-abc4-a8822b17daca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '522ad1fff47a406fb97ac7f1d1fc920e'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:21.369040 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-d0872cf3-727f-4145-abc4-a8822b17daca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '522ad1fff47a406fb97ac7f1d1fc920e'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:21.376402 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1439/2889] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:10:21 2022] DELETE /identity/v3/users/522ad1fff47a406fb97ac7f1d1fc920e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:21.656546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3923b2-e107-4042-96ed-a24c989320fc admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:21.658065 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-ee3923b2-e107-4042-96ed-a24c989320fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:21.672569 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3923b2-e107-4042-96ed-a24c989320fc admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:21.739634 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d41b3-a311-4191-bf99-0be3d7827357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:21.740778 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d41b3-a311-4191-bf99-0be3d7827357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:21.740778 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d41b3-a311-4191-bf99-0be3d7827357 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:21.741031 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d41b3-a311-4191-bf99-0be3d7827357 admin admin] PATH_INFO: `/v3/projects/db6371b090764f7296f86cd1baa66e28` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:21.742658 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d41b3-a311-4191-bf99-0be3d7827357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db6371b090764f7296f86cd1baa66e28)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:21.744630 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d41b3-a311-4191-bf99-0be3d7827357 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:21.790729 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-199d41b3-a311-4191-bf99-0be3d7827357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db6371b090764f7296f86cd1baa66e28', 'request_id': 'req-199d41b3-a311-4191-bf99-0be3d7827357'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:21.796130 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-199d41b3-a311-4191-bf99-0be3d7827357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db6371b090764f7296f86cd1baa66e28', 'request_id': 'req-199d41b3-a311-4191-bf99-0be3d7827357'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:21.803526 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-199d41b3-a311-4191-bf99-0be3d7827357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db6371b090764f7296f86cd1baa66e28', 'request_id': 'req-199d41b3-a311-4191-bf99-0be3d7827357'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:21.815661 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-199d41b3-a311-4191-bf99-0be3d7827357 admin admin] The token cache is being invalidate because project db6371b090764f7296f86cd1baa66e28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:10:21.816041 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-199d41b3-a311-4191-bf99-0be3d7827357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:21.816716 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-199d41b3-a311-4191-bf99-0be3d7827357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:21.818230 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1451/2890] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:10:21 2022] DELETE /identity/v3/projects/db6371b090764f7296f86cd1baa66e28 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:21.823379 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4f47d-7192-4f07-a6e1-fc8947814b85 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:21.823891 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4f47d-7192-4f07-a6e1-fc8947814b85 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:21.824009 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4f47d-7192-4f07-a6e1-fc8947814b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:21.843815 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-faf4f47d-7192-4f07-a6e1-fc8947814b85 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:21.844970 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-faf4f47d-7192-4f07-a6e1-fc8947814b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:21.903972 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1440/2891] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:10:21.911670 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f561fa2-7af7-4fc3-a358-45d70737f502 None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:21.911958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f561fa2-7af7-4fc3-a358-45d70737f502 None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:21.912237 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f561fa2-7af7-4fc3-a358-45d70737f502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:21.932957 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-4f561fa2-7af7-4fc3-a358-45d70737f502 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:21.934187 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-4f561fa2-7af7-4fc3-a358-45d70737f502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:21.942350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1452/2892] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:10:21.946666 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4f47d-7192-4f07-a6e1-fc8947814b85 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:21.952780 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:21.956847 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4f47d-7192-4f07-a6e1-fc8947814b85 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:21.963474 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf6536-c846-4075-a46c-aebde252ef89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:21.964330 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf6536-c846-4075-a46c-aebde252ef89 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:21.964902 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf6536-c846-4075-a46c-aebde252ef89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:21.965311 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf6536-c846-4075-a46c-aebde252ef89 admin admin] PATH_INFO: `/v3/domains` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:21.966135 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf6536-c846-4075-a46c-aebde252ef89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91894) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 19:10:21.966651 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf6536-c846-4075-a46c-aebde252ef89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:21.968524 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf6536-c846-4075-a46c-aebde252ef89 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:21.981147 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1441/2893] 10.222.0.69 () {58 vars in 1153 bytes} [Wed Aug 3 19:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 19:10:21.985504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f561fa2-7af7-4fc3-a358-45d70737f502 None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:21.995308 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f561fa2-7af7-4fc3-a358-45d70737f502 None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:21.999989 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbd8cb-e0ae-4d1d-b824-f3f7b0ab7ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:22.000628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbd8cb-e0ae-4d1d-b824-f3f7b0ab7ecb admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:22.000913 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbd8cb-e0ae-4d1d-b824-f3f7b0ab7ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:22.001214 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbd8cb-e0ae-4d1d-b824-f3f7b0ab7ecb admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:22.001977 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbd8cb-e0ae-4d1d-b824-f3f7b0ab7ecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:22.003871 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbd8cb-e0ae-4d1d-b824-f3f7b0ab7ecb admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:22.031741 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1453/2894] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:10:21 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 19:10:22.035751 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf6536-c846-4075-a46c-aebde252ef89 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:22.046794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf6536-c846-4075-a46c-aebde252ef89 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:22.103389 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca6395-d4d7-426f-b7e6-26e3589792f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:22.103927 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca6395-d4d7-426f-b7e6-26e3589792f8 admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:22.104210 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca6395-d4d7-426f-b7e6-26e3589792f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:22.104513 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca6395-d4d7-426f-b7e6-26e3589792f8 admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:22.105186 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca6395-d4d7-426f-b7e6-26e3589792f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:22.107085 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca6395-d4d7-426f-b7e6-26e3589792f8 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:22.127119 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1442/2895] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:10:22 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:10:22.131262 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbd8cb-e0ae-4d1d-b824-f3f7b0ab7ecb admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:22.142868 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbd8cb-e0ae-4d1d-b824-f3f7b0ab7ecb admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:22.147473 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3f1d9-df03-4013-848e-7dc6c086af06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:22.148840 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3f1d9-df03-4013-848e-7dc6c086af06 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:22.149088 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3f1d9-df03-4013-848e-7dc6c086af06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:22.149363 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3f1d9-df03-4013-848e-7dc6c086af06 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:22.150063 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3f1d9-df03-4013-848e-7dc6c086af06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:22.151661 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3f1d9-df03-4013-848e-7dc6c086af06 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:22.161676 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1454/2896] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:10:22.165918 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca6395-d4d7-426f-b7e6-26e3589792f8 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:22.175809 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca6395-d4d7-426f-b7e6-26e3589792f8 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:22.180941 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a40206-73b1-4198-8268-574a4e285325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:22.181549 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a40206-73b1-4198-8268-574a4e285325 admin admin] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:22.181829 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a40206-73b1-4198-8268-574a4e285325 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:22.182101 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a40206-73b1-4198-8268-574a4e285325 admin admin] PATH_INFO: `/v3/roles` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:22.182791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a40206-73b1-4198-8268-574a4e285325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:22.184400 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a40206-73b1-4198-8268-574a4e285325 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:22.198831 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1443/2897] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:10:22.202971 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3f1d9-df03-4013-848e-7dc6c086af06 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:22.213782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3f1d9-df03-4013-848e-7dc6c086af06 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:22.218431 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b149085-35c5-42ba-9c5c-a9e4f9e1a02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:22.219108 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b149085-35c5-42ba-9c5c-a9e4f9e1a02e admin admin] REQUEST_METHOD: `PUT` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:22.219378 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b149085-35c5-42ba-9c5c-a9e4f9e1a02e admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:22.219655 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b149085-35c5-42ba-9c5c-a9e4f9e1a02e admin admin] PATH_INFO: `/v3/projects/b26f9a80b1924da093f0082df7dd06ee/users/5bb6e0fadde3457cb8b93f92b012de83/roles/f3b7ffe4d1f04af18f7da8d9a51031b3` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:22.234234 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b149085-35c5-42ba-9c5c-a9e4f9e1a02e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b26f9a80b1924da093f0082df7dd06ee, user_id=5bb6e0fadde3457cb8b93f92b012de83, role_id=f3b7ffe4d1f04af18f7da8d9a51031b3)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:22.236937 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b149085-35c5-42ba-9c5c-a9e4f9e1a02e admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:22.249524 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1455/2898] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:10:22 2022] PUT /identity/v3/projects/b26f9a80b1924da093f0082df7dd06ee/users/5bb6e0fadde3457cb8b93f92b012de83/roles/f3b7ffe4d1f04af18f7da8d9a51031b3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:22.261916 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d035d4c-7349-44d7-8994-ccc3c09413f4 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:22.262201 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d035d4c-7349-44d7-8994-ccc3c09413f4 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:22.262477 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d035d4c-7349-44d7-8994-ccc3c09413f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:22.290543 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-7d035d4c-7349-44d7-8994-ccc3c09413f4 None None] MFA Rules not processed for user `5bb6e0fadde3457cb8b93f92b012de83`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:22.291755 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-7d035d4c-7349-44d7-8994-ccc3c09413f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:22.357791 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1444/2899] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:10:22.362248 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b149085-35c5-42ba-9c5c-a9e4f9e1a02e admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:22.372412 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:22.427567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b149085-35c5-42ba-9c5c-a9e4f9e1a02e admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:22.433371 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37748a10-2dd0-4d4d-9238-6956c325a897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:22.433986 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37748a10-2dd0-4d4d-9238-6956c325a897 admin admin] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:22.434265 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37748a10-2dd0-4d4d-9238-6956c325a897 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:22.434562 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37748a10-2dd0-4d4d-9238-6956c325a897 admin admin] PATH_INFO: `/v3/projects` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:22.435233 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37748a10-2dd0-4d4d-9238-6956c325a897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:22.437167 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37748a10-2dd0-4d4d-9238-6956c325a897 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:22.465854 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1456/2900] 10.222.0.69 () {60 vars in 1153 bytes} [Wed Aug 3 19:10:22 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:10:22.469463 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d035d4c-7349-44d7-8994-ccc3c09413f4 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:22.480987 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d035d4c-7349-44d7-8994-ccc3c09413f4 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:22.537016 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7680f7-c76f-4fdf-900c-26b29bca424c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:22.537638 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7680f7-c76f-4fdf-900c-26b29bca424c admin admin] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:22.537924 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7680f7-c76f-4fdf-900c-26b29bca424c admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:22.538201 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7680f7-c76f-4fdf-900c-26b29bca424c admin admin] PATH_INFO: `/v3/users` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:22.539636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7680f7-c76f-4fdf-900c-26b29bca424c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:22.541571 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7680f7-c76f-4fdf-900c-26b29bca424c admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:22.549374 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:22.565351 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1445/2901] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Aug 3 19:10:22 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 19:10:22.569428 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37748a10-2dd0-4d4d-9238-6956c325a897 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:22.580958 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37748a10-2dd0-4d4d-9238-6956c325a897 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:22.585647 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf82f8-a63d-40a3-9375-71540cb4ea00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:22.586264 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf82f8-a63d-40a3-9375-71540cb4ea00 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:22.586549 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf82f8-a63d-40a3-9375-71540cb4ea00 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:22.586826 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf82f8-a63d-40a3-9375-71540cb4ea00 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:22.587526 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf82f8-a63d-40a3-9375-71540cb4ea00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:22.589163 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf82f8-a63d-40a3-9375-71540cb4ea00 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:22.599731 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1457/2902] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:10:22.604599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7680f7-c76f-4fdf-900c-26b29bca424c admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:22.614636 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7680f7-c76f-4fdf-900c-26b29bca424c admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:22.620084 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61904f42-184c-4e6b-8ae7-2af9aae04c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:22.620782 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61904f42-184c-4e6b-8ae7-2af9aae04c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:22.621067 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61904f42-184c-4e6b-8ae7-2af9aae04c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:22.621345 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61904f42-184c-4e6b-8ae7-2af9aae04c84 admin admin] PATH_INFO: `/v3/projects/eb71614444524c789b25253fa9937eb5/users/8242fd2c8fb6401ca296445b2c6b2a47/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:22.636520 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61904f42-184c-4e6b-8ae7-2af9aae04c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb71614444524c789b25253fa9937eb5, user_id=8242fd2c8fb6401ca296445b2c6b2a47, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:22.639200 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61904f42-184c-4e6b-8ae7-2af9aae04c84 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:22.652544 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1446/2903] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:10:22 2022] PUT /identity/v3/projects/eb71614444524c789b25253fa9937eb5/users/8242fd2c8fb6401ca296445b2c6b2a47/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:22.656567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf82f8-a63d-40a3-9375-71540cb4ea00 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:22.668027 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf82f8-a63d-40a3-9375-71540cb4ea00 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:22.726961 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59dd6b-f956-4738-8002-5c8c7c6c3e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:22.727718 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59dd6b-f956-4738-8002-5c8c7c6c3e60 admin admin] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:22.727877 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59dd6b-f956-4738-8002-5c8c7c6c3e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:22.728153 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59dd6b-f956-4738-8002-5c8c7c6c3e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:22.728881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59dd6b-f956-4738-8002-5c8c7c6c3e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:22.730513 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59dd6b-f956-4738-8002-5c8c7c6c3e60 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:22.740567 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1458/2904] 10.222.0.69 () {58 vars in 1122 bytes} [Wed Aug 3 19:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 19:10:22.745867 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61904f42-184c-4e6b-8ae7-2af9aae04c84 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:22.756204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61904f42-184c-4e6b-8ae7-2af9aae04c84 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:22.761654 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f9d2a-d708-4f88-adce-5b987cdea246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:22.762319 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f9d2a-d708-4f88-adce-5b987cdea246 admin admin] REQUEST_METHOD: `PUT` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:22.762606 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f9d2a-d708-4f88-adce-5b987cdea246 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:22.762879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f9d2a-d708-4f88-adce-5b987cdea246 admin admin] PATH_INFO: `/v3/projects/eb71614444524c789b25253fa9937eb5/users/8242fd2c8fb6401ca296445b2c6b2a47/roles/19557e941d6f475ba92cf862846f9872` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:22.764860 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f9d2a-d708-4f88-adce-5b987cdea246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb71614444524c789b25253fa9937eb5, user_id=8242fd2c8fb6401ca296445b2c6b2a47, role_id=19557e941d6f475ba92cf862846f9872)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:22.767500 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f9d2a-d708-4f88-adce-5b987cdea246 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:22.777826 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1447/2905] 10.222.0.69 () {60 vars in 1483 bytes} [Wed Aug 3 19:10:22 2022] PUT /identity/v3/projects/eb71614444524c789b25253fa9937eb5/users/8242fd2c8fb6401ca296445b2c6b2a47/roles/19557e941d6f475ba92cf862846f9872 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:22.787881 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbb77d-a020-4db2-9257-3ccc052980cb None None] REQUEST_METHOD: `POST` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:22.788171 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbb77d-a020-4db2-9257-3ccc052980cb None None] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:22.789139 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbb77d-a020-4db2-9257-3ccc052980cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:22.818468 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.auth.core [None req-eabbb77d-a020-4db2-9257-3ccc052980cb None None] MFA Rules not processed for user `8242fd2c8fb6401ca296445b2c6b2a47`. Rule list: `[]` (Enabled: `True`). {{(pid=91895) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:22.819798 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-eabbb77d-a020-4db2-9257-3ccc052980cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:22.888722 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1459/2906] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:10:22.900228 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f9d2a-d708-4f88-adce-5b987cdea246 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:22.902018 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-b38f9d2a-d708-4f88-adce-5b987cdea246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:22.961204 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f9d2a-d708-4f88-adce-5b987cdea246 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:22.964857 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904642a8-96d8-4601-b3e0-56d71d55a585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d0fde945184c93b8fd91d15b66f534', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:22.965472 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904642a8-96d8-4601-b3e0-56d71d55a585 service cinder] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:22.965752 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904642a8-96d8-4601-b3e0-56d71d55a585 service cinder] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:22.966025 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904642a8-96d8-4601-b3e0-56d71d55a585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:23.022361 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904642a8-96d8-4601-b3e0-56d71d55a585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:23.024037 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904642a8-96d8-4601-b3e0-56d71d55a585 service cinder] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:23.033060 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1448/2907] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:10:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:10:25.570559 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbb77d-a020-4db2-9257-3ccc052980cb None None] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:25.574849 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:25.636944 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbb77d-a020-4db2-9257-3ccc052980cb None None] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:25.641546 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19b40b-f093-4aa9-b4c7-2112d5f96787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:25.642199 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19b40b-f093-4aa9-b4c7-2112d5f96787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:25.642481 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19b40b-f093-4aa9-b4c7-2112d5f96787 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:25.642761 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19b40b-f093-4aa9-b4c7-2112d5f96787 admin admin] PATH_INFO: `/v3/users/5bb6e0fadde3457cb8b93f92b012de83` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:25.643930 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19b40b-f093-4aa9-b4c7-2112d5f96787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bb6e0fadde3457cb8b93f92b012de83)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:25.645902 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19b40b-f093-4aa9-b4c7-2112d5f96787 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:25.691506 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8b19b40b-f093-4aa9-b4c7-2112d5f96787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bb6e0fadde3457cb8b93f92b012de83'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:25.699840 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8b19b40b-f093-4aa9-b4c7-2112d5f96787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bb6e0fadde3457cb8b93f92b012de83'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:25.712540 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8b19b40b-f093-4aa9-b4c7-2112d5f96787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bb6e0fadde3457cb8b93f92b012de83'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:25.719344 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8b19b40b-f093-4aa9-b4c7-2112d5f96787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bb6e0fadde3457cb8b93f92b012de83'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:25.719972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-8b19b40b-f093-4aa9-b4c7-2112d5f96787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bb6e0fadde3457cb8b93f92b012de83'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:25.727002 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1460/2908] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:10:25 2022] DELETE /identity/v3/users/5bb6e0fadde3457cb8b93f92b012de83 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:25.731528 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e542e40-25e8-4dd4-918d-b06fb0fbc249 None None] REQUEST_METHOD: `POST` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:25.732043 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e542e40-25e8-4dd4-918d-b06fb0fbc249 None None] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:25.732376 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e542e40-25e8-4dd4-918d-b06fb0fbc249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:25.751551 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.auth.core [None req-4e542e40-25e8-4dd4-918d-b06fb0fbc249 None None] MFA Rules not processed for user `b499f31220244d41877273ff02e31ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=91894) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 19:10:25.752778 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4e542e40-25e8-4dd4-918d-b06fb0fbc249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:25.810860 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1449/2909] 10.222.0.69 () {58 vars in 958 bytes} [Wed Aug 3 19:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 19:10:25.820795 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19b40b-f093-4aa9-b4c7-2112d5f96787 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:25.822193 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-8b19b40b-f093-4aa9-b4c7-2112d5f96787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:25.881004 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19b40b-f093-4aa9-b4c7-2112d5f96787 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:25.885354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8919ecc-cbcd-4719-95b4-b602595ef0b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:25.885972 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8919ecc-cbcd-4719-95b4-b602595ef0b1 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:25.886505 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8919ecc-cbcd-4719-95b4-b602595ef0b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:25.886779 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8919ecc-cbcd-4719-95b4-b602595ef0b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:25.895833 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8919ecc-cbcd-4719-95b4-b602595ef0b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:25.897296 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8919ecc-cbcd-4719-95b4-b602595ef0b1 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:25.907261 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1461/2910] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:10:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:10:26.180680 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e542e40-25e8-4dd4-918d-b06fb0fbc249 None None] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:26.182859 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-4e542e40-25e8-4dd4-918d-b06fb0fbc249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:26.195505 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e542e40-25e8-4dd4-918d-b06fb0fbc249 None None] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:26.200277 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19012d-6390-444f-8834-affc3d50f446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:26.200965 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19012d-6390-444f-8834-affc3d50f446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:26.201252 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19012d-6390-444f-8834-affc3d50f446 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:26.201527 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19012d-6390-444f-8834-affc3d50f446 admin admin] PATH_INFO: `/v3/projects/b26f9a80b1924da093f0082df7dd06ee` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:26.202707 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19012d-6390-444f-8834-affc3d50f446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b26f9a80b1924da093f0082df7dd06ee)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:26.204661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19012d-6390-444f-8834-affc3d50f446 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:26.206309 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:26.256364 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1a19012d-6390-444f-8834-affc3d50f446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b26f9a80b1924da093f0082df7dd06ee', 'request_id': 'req-1a19012d-6390-444f-8834-affc3d50f446'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:26.263154 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1a19012d-6390-444f-8834-affc3d50f446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b26f9a80b1924da093f0082df7dd06ee', 'request_id': 'req-1a19012d-6390-444f-8834-affc3d50f446'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:26.269357 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1a19012d-6390-444f-8834-affc3d50f446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b26f9a80b1924da093f0082df7dd06ee', 'request_id': 'req-1a19012d-6390-444f-8834-affc3d50f446'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:26.283410 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1a19012d-6390-444f-8834-affc3d50f446 admin admin] The token cache is being invalidate because project b26f9a80b1924da093f0082df7dd06ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:10:26.283699 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1a19012d-6390-444f-8834-affc3d50f446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:26.284353 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-1a19012d-6390-444f-8834-affc3d50f446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:26.285855 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1450/2911] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:10:26 2022] DELETE /identity/v3/projects/b26f9a80b1924da093f0082df7dd06ee => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:26.290337 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8919ecc-cbcd-4719-95b4-b602595ef0b1 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:26.299917 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8919ecc-cbcd-4719-95b4-b602595ef0b1 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:26.319215 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:26.359040 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad03c42-2ddf-4f07-ab83-78ffa7459dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:26.359598 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad03c42-2ddf-4f07-ab83-78ffa7459dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:26.359888 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad03c42-2ddf-4f07-ab83-78ffa7459dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:26.360157 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad03c42-2ddf-4f07-ab83-78ffa7459dcc admin admin] PATH_INFO: `/v3/users/8242fd2c8fb6401ca296445b2c6b2a47` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:26.361353 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad03c42-2ddf-4f07-ab83-78ffa7459dcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8242fd2c8fb6401ca296445b2c6b2a47)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:26.363286 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad03c42-2ddf-4f07-ab83-78ffa7459dcc admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:26.413659 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-7ad03c42-2ddf-4f07-ab83-78ffa7459dcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8242fd2c8fb6401ca296445b2c6b2a47'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:26.420969 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-7ad03c42-2ddf-4f07-ab83-78ffa7459dcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8242fd2c8fb6401ca296445b2c6b2a47'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:26.433176 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-7ad03c42-2ddf-4f07-ab83-78ffa7459dcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8242fd2c8fb6401ca296445b2c6b2a47'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:26.440839 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-7ad03c42-2ddf-4f07-ab83-78ffa7459dcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8242fd2c8fb6401ca296445b2c6b2a47'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:26.441493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-7ad03c42-2ddf-4f07-ab83-78ffa7459dcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8242fd2c8fb6401ca296445b2c6b2a47'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:26.447993 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1462/2912] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:10:26 2022] DELETE /identity/v3/users/8242fd2c8fb6401ca296445b2c6b2a47 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:26.727801 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19012d-6390-444f-8834-affc3d50f446 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:10:26.730305 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1a19012d-6390-444f-8834-affc3d50f446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:10:26.745914 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19012d-6390-444f-8834-affc3d50f446 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:10:26.806541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd43ce-18c4-4363-b5c9-fec286799e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:10:26.807188 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd43ce-18c4-4363-b5c9-fec286799e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:10:26.807496 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd43ce-18c4-4363-b5c9-fec286799e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:10:26.807766 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd43ce-18c4-4363-b5c9-fec286799e22 admin admin] PATH_INFO: `/v3/projects/eb71614444524c789b25253fa9937eb5` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:10:26.808951 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd43ce-18c4-4363-b5c9-fec286799e22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb71614444524c789b25253fa9937eb5)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:10:26.810850 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd43ce-18c4-4363-b5c9-fec286799e22 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:10:26.859740 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bdfd43ce-18c4-4363-b5c9-fec286799e22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb71614444524c789b25253fa9937eb5', 'request_id': 'req-bdfd43ce-18c4-4363-b5c9-fec286799e22'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:26.865134 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bdfd43ce-18c4-4363-b5c9-fec286799e22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb71614444524c789b25253fa9937eb5', 'request_id': 'req-bdfd43ce-18c4-4363-b5c9-fec286799e22'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:26.872248 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bdfd43ce-18c4-4363-b5c9-fec286799e22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb71614444524c789b25253fa9937eb5', 'request_id': 'req-bdfd43ce-18c4-4363-b5c9-fec286799e22'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:26.885363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bdfd43ce-18c4-4363-b5c9-fec286799e22 admin admin] The token cache is being invalidate because project eb71614444524c789b25253fa9937eb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91894) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:10:26.885652 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bdfd43ce-18c4-4363-b5c9-fec286799e22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:26.886535 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-bdfd43ce-18c4-4363-b5c9-fec286799e22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:10:26.888040 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1451/2913] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:10:26 2022] DELETE /identity/v3/projects/eb71614444524c789b25253fa9937eb5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:10:36.443880 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:10:36.881393 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=75,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:11:45.951112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad03c42-2ddf-4f07-ab83-78ffa7459dcc admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:11:45.952579 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-7ad03c42-2ddf-4f07-ab83-78ffa7459dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:11:46.010099 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad03c42-2ddf-4f07-ab83-78ffa7459dcc admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:11:46.061609 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d45b64-698c-4324-b45b-49fae2ef2e5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:11:46.062227 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d45b64-698c-4324-b45b-49fae2ef2e5d service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:11:46.062503 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d45b64-698c-4324-b45b-49fae2ef2e5d service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:11:46.062773 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d45b64-698c-4324-b45b-49fae2ef2e5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:11:46.064583 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-74d45b64-698c-4324-b45b-49fae2ef2e5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:11:46.074406 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d45b64-698c-4324-b45b-49fae2ef2e5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:11:46.075829 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d45b64-698c-4324-b45b-49fae2ef2e5d service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:11:46.083646 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1463/2914] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:11:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:11:56.080148 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=46 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:12:17.465295 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd43ce-18c4-4363-b5c9-fec286799e22 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:12:17.467157 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-bdfd43ce-18c4-4363-b5c9-fec286799e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:12:17.557566 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd43ce-18c4-4363-b5c9-fec286799e22 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:12:17.565207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d4eee-d955-4999-9b26-82a88af4fb5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '850a50cfe4394febbafb0908b08888b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:12:17.565207 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d4eee-d955-4999-9b26-82a88af4fb5c service nova] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:12:17.565599 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d4eee-d955-4999-9b26-82a88af4fb5c service nova] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:12:17.565950 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d4eee-d955-4999-9b26-82a88af4fb5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:12:17.567879 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-a12d4eee-d955-4999-9b26-82a88af4fb5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:12:17.582019 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d4eee-d955-4999-9b26-82a88af4fb5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:12:17.583629 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d4eee-d955-4999-9b26-82a88af4fb5c service nova] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:12:17.602183 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1452/2915] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:12:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:12:17.651447 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d45b64-698c-4324-b45b-49fae2ef2e5d service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:12:17.652908 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-74d45b64-698c-4324-b45b-49fae2ef2e5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:12:17.663423 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d45b64-698c-4324-b45b-49fae2ef2e5d service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:12:17.668485 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb99b1-21b0-460d-96db-875fcaf0f613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:12:17.669122 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb99b1-21b0-460d-96db-875fcaf0f613 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:12:17.669404 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb99b1-21b0-460d-96db-875fcaf0f613 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:12:17.669680 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb99b1-21b0-460d-96db-875fcaf0f613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:12:17.671433 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-04eb99b1-21b0-460d-96db-875fcaf0f613 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:12:17.686570 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb99b1-21b0-460d-96db-875fcaf0f613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:12:17.688014 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb99b1-21b0-460d-96db-875fcaf0f613 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:12:17.699846 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1464/2916] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:12:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:12:18.887322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d4eee-d955-4999-9b26-82a88af4fb5c service nova] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:12:18.897038 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d4eee-d955-4999-9b26-82a88af4fb5c service nova] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:12:18.900694 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449427ed-5f0a-47c1-8127-2a7f727064d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:12:18.901491 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449427ed-5f0a-47c1-8127-2a7f727064d1 service neutron] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:12:18.901772 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449427ed-5f0a-47c1-8127-2a7f727064d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:12:18.902048 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449427ed-5f0a-47c1-8127-2a7f727064d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:12:18.903838 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-449427ed-5f0a-47c1-8127-2a7f727064d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:12:18.918230 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449427ed-5f0a-47c1-8127-2a7f727064d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:12:18.919670 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449427ed-5f0a-47c1-8127-2a7f727064d1 service neutron] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:12:18.932955 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1453/2917] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:12:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:12:27.694336 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=6 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:12:28.926908 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=40 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:12:30.101847 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb99b1-21b0-460d-96db-875fcaf0f613 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:12:30.111096 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb99b1-21b0-460d-96db-875fcaf0f613 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:12:30.114782 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f3fbe-2476-492f-9bdf-a49dd38ebdc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0eb9ad672e5444098d71b7c9a27b72e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:12:30.115392 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f3fbe-2476-492f-9bdf-a49dd38ebdc1 service neutron] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:12:30.115683 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f3fbe-2476-492f-9bdf-a49dd38ebdc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:12:30.115951 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f3fbe-2476-492f-9bdf-a49dd38ebdc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:12:30.117766 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-d47f3fbe-2476-492f-9bdf-a49dd38ebdc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:12:30.183112 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f3fbe-2476-492f-9bdf-a49dd38ebdc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:12:30.184586 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f3fbe-2476-492f-9bdf-a49dd38ebdc1 service neutron] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:12:30.194152 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1465/2918] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:12:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:12:33.373686 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449427ed-5f0a-47c1-8127-2a7f727064d1 service neutron] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:12:33.375164 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-449427ed-5f0a-47c1-8127-2a7f727064d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:12:33.475288 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449427ed-5f0a-47c1-8127-2a7f727064d1 service neutron] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:12:33.481461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc54c8d-f320-43f8-9458-b827b25be2c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:12:33.482105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc54c8d-f320-43f8-9458-b827b25be2c3 service placement] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:12:33.482386 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc54c8d-f320-43f8-9458-b827b25be2c3 service placement] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:12:33.482661 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc54c8d-f320-43f8-9458-b827b25be2c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:12:33.485153 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1dc54c8d-f320-43f8-9458-b827b25be2c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:12:33.499530 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc54c8d-f320-43f8-9458-b827b25be2c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:12:33.501322 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc54c8d-f320-43f8-9458-b827b25be2c3 service placement] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:12:33.513261 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1454/2919] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:12:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:12:36.568632 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f3fbe-2476-492f-9bdf-a49dd38ebdc1 service neutron] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:12:36.581879 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f3fbe-2476-492f-9bdf-a49dd38ebdc1 service neutron] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:12:36.587260 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f3344-e15d-48e0-903d-117ae5b5127b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:12:36.587967 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f3344-e15d-48e0-903d-117ae5b5127b service placement] REQUEST_METHOD: `GET` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:12:36.588339 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f3344-e15d-48e0-903d-117ae5b5127b service placement] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:12:36.588689 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f3344-e15d-48e0-903d-117ae5b5127b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:12:36.590604 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-223f3344-e15d-48e0-903d-117ae5b5127b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:12:36.606616 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f3344-e15d-48e0-903d-117ae5b5127b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:12:36.608504 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f3344-e15d-48e0-903d-117ae5b5127b service placement] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:12:36.628885 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1466/2920] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:12:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 19:12:36.630005 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc54c8d-f320-43f8-9458-b827b25be2c3 service placement] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:12:36.632048 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-1dc54c8d-f320-43f8-9458-b827b25be2c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:12:36.647832 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc54c8d-f320-43f8-9458-b827b25be2c3 service placement] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:12:36.654226 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498093b2-3bf5-46db-8b31-65ebf2527980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:12:36.654889 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498093b2-3bf5-46db-8b31-65ebf2527980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:12:36.655179 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498093b2-3bf5-46db-8b31-65ebf2527980 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:12:36.655458 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498093b2-3bf5-46db-8b31-65ebf2527980 admin admin] PATH_INFO: `/v3/users/4f7e62d82d054b98bd22dc6958577a74` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:12:36.656804 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498093b2-3bf5-46db-8b31-65ebf2527980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f7e62d82d054b98bd22dc6958577a74)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:12:36.658854 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498093b2-3bf5-46db-8b31-65ebf2527980 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:12:36.711092 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-498093b2-3bf5-46db-8b31-65ebf2527980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f7e62d82d054b98bd22dc6958577a74'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:36.719989 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-498093b2-3bf5-46db-8b31-65ebf2527980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f7e62d82d054b98bd22dc6958577a74'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:36.736780 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-498093b2-3bf5-46db-8b31-65ebf2527980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f7e62d82d054b98bd22dc6958577a74'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:36.746423 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-498093b2-3bf5-46db-8b31-65ebf2527980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f7e62d82d054b98bd22dc6958577a74'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:36.747171 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-498093b2-3bf5-46db-8b31-65ebf2527980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f7e62d82d054b98bd22dc6958577a74'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:36.754461 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1455/2921] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:12:36 2022] DELETE /identity/v3/users/4f7e62d82d054b98bd22dc6958577a74 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:12:36.957675 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f3344-e15d-48e0-903d-117ae5b5127b service placement] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:12:36.975021 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f3344-e15d-48e0-903d-117ae5b5127b service placement] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:12:37.053403 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b67e4-2098-4beb-a543-d41298cb0ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:12:37.054062 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b67e4-2098-4beb-a543-d41298cb0ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:12:37.054350 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b67e4-2098-4beb-a543-d41298cb0ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:12:37.054628 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b67e4-2098-4beb-a543-d41298cb0ea2 admin admin] PATH_INFO: `/v3/projects/7041552fa8cc4a11ae098f1d2a16e359` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:12:37.055828 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b67e4-2098-4beb-a543-d41298cb0ea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7041552fa8cc4a11ae098f1d2a16e359)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:12:37.058558 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b67e4-2098-4beb-a543-d41298cb0ea2 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:12:37.114758 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-231b67e4-2098-4beb-a543-d41298cb0ea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7041552fa8cc4a11ae098f1d2a16e359', 'request_id': 'req-231b67e4-2098-4beb-a543-d41298cb0ea2'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:37.122480 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-231b67e4-2098-4beb-a543-d41298cb0ea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7041552fa8cc4a11ae098f1d2a16e359', 'request_id': 'req-231b67e4-2098-4beb-a543-d41298cb0ea2'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:37.133878 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-231b67e4-2098-4beb-a543-d41298cb0ea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7041552fa8cc4a11ae098f1d2a16e359', 'request_id': 'req-231b67e4-2098-4beb-a543-d41298cb0ea2'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:37.147754 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-231b67e4-2098-4beb-a543-d41298cb0ea2 admin admin] The token cache is being invalidate because project 7041552fa8cc4a11ae098f1d2a16e359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:12:37.148045 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-231b67e4-2098-4beb-a543-d41298cb0ea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:37.148783 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-231b67e4-2098-4beb-a543-d41298cb0ea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:37.150948 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1467/2922] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:12:36 2022] DELETE /identity/v3/projects/7041552fa8cc4a11ae098f1d2a16e359 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:12:37.162072 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498093b2-3bf5-46db-8b31-65ebf2527980 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:12:37.163733 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-498093b2-3bf5-46db-8b31-65ebf2527980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:12:37.180899 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498093b2-3bf5-46db-8b31-65ebf2527980 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:12:37.263373 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e5e8e-ec35-4838-8c1e-9cfa77b0b721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:12:37.264237 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e5e8e-ec35-4838-8c1e-9cfa77b0b721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:12:37.264612 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e5e8e-ec35-4838-8c1e-9cfa77b0b721 admin admin] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:12:37.265023 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e5e8e-ec35-4838-8c1e-9cfa77b0b721 admin admin] PATH_INFO: `/v3/users/073c1605dd43489d83472c7523d65bf9` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:12:37.266375 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e5e8e-ec35-4838-8c1e-9cfa77b0b721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=073c1605dd43489d83472c7523d65bf9)` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:12:37.268494 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e5e8e-ec35-4838-8c1e-9cfa77b0b721 admin admin] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:12:37.271137 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:12:37.319105 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-843e5e8e-ec35-4838-8c1e-9cfa77b0b721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '073c1605dd43489d83472c7523d65bf9'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:37.326794 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-843e5e8e-ec35-4838-8c1e-9cfa77b0b721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '073c1605dd43489d83472c7523d65bf9'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:37.340126 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-843e5e8e-ec35-4838-8c1e-9cfa77b0b721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '073c1605dd43489d83472c7523d65bf9'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:37.347238 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-843e5e8e-ec35-4838-8c1e-9cfa77b0b721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '073c1605dd43489d83472c7523d65bf9'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:37.348099 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.notifications [None req-843e5e8e-ec35-4838-8c1e-9cfa77b0b721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '073c1605dd43489d83472c7523d65bf9'} {{(pid=91894) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:37.357363 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1456/2923] 10.222.0.69 () {58 vars in 1224 bytes} [Wed Aug 3 19:12:37 2022] DELETE /identity/v3/users/073c1605dd43489d83472c7523d65bf9 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:12:37.537145 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b67e4-2098-4beb-a543-d41298cb0ea2 admin admin] Authenticating user token {{(pid=91895) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:12:37.538581 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.fernet_utils [None req-231b67e4-2098-4beb-a543-d41298cb0ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91895) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:12:37.559398 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b67e4-2098-4beb-a543-d41298cb0ea2 admin admin] Validating token access rules against request {{(pid=91895) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:12:37.568302 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:12:37.618493 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b6196-a913-4e66-98ca-0494cfc164d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b499f31220244d41877273ff02e31ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649429edefe44474b26fed9c93d5ec67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91895) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:12:37.619325 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b6196-a913-4e66-98ca-0494cfc164d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:12:37.619532 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b6196-a913-4e66-98ca-0494cfc164d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:12:37.619818 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b6196-a913-4e66-98ca-0494cfc164d5 admin admin] PATH_INFO: `/v3/projects/81fce08714e64e22a0c6466499829fdb` {{(pid=91895) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:12:37.621028 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b6196-a913-4e66-98ca-0494cfc164d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81fce08714e64e22a0c6466499829fdb)` {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:12:37.622955 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b6196-a913-4e66-98ca-0494cfc164d5 admin admin] RBAC: Authorization granted {{(pid=91895) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:12:37.670774 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-291b6196-a913-4e66-98ca-0494cfc164d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81fce08714e64e22a0c6466499829fdb', 'request_id': 'req-291b6196-a913-4e66-98ca-0494cfc164d5'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:37.676179 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-291b6196-a913-4e66-98ca-0494cfc164d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81fce08714e64e22a0c6466499829fdb', 'request_id': 'req-291b6196-a913-4e66-98ca-0494cfc164d5'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:37.691354 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-291b6196-a913-4e66-98ca-0494cfc164d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81fce08714e64e22a0c6466499829fdb', 'request_id': 'req-291b6196-a913-4e66-98ca-0494cfc164d5'} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:37.704529 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-291b6196-a913-4e66-98ca-0494cfc164d5 admin admin] The token cache is being invalidate because project 81fce08714e64e22a0c6466499829fdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91895) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 19:12:37.704820 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-291b6196-a913-4e66-98ca-0494cfc164d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:37.705471 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG keystone.notifications [None req-291b6196-a913-4e66-98ca-0494cfc164d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91895) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 19:12:37.706978 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: [pid: 91895|app: 0|req: 1468/2924] 10.222.0.69 () {58 vars in 1233 bytes} [Wed Aug 3 19:12:37 2022] DELETE /identity/v3/projects/81fce08714e64e22a0c6466499829fdb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 19:12:47.351866 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:12:47.700539 dv-cinder-851640-2-iscsi devstack@keystone.service[91895]: DEBUG dbcounter [-] [91895] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91895) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 19:14:46.546094 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e5e8e-ec35-4838-8c1e-9cfa77b0b721 admin admin] Authenticating user token {{(pid=91894) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 19:14:46.547462 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-843e5e8e-ec35-4838-8c1e-9cfa77b0b721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:14:46.619541 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e5e8e-ec35-4838-8c1e-9cfa77b0b721 admin admin] Validating token access rules against request {{(pid=91894) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 19:14:46.680316 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f57c4-24b1-4f96-84b6-9a5333b92ced service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fea93ceb4cbc49c2b4b280a4ddc257b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59f00389bb404b4392d98e22e045c84f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91894) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 19:14:46.680957 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f57c4-24b1-4f96-84b6-9a5333b92ced service placement] REQUEST_METHOD: `GET` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 19:14:46.681240 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f57c4-24b1-4f96-84b6-9a5333b92ced service placement] SCRIPT_NAME: `/identity` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 19:14:46.681514 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f57c4-24b1-4f96-84b6-9a5333b92ced service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91894) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 19:14:46.683278 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.fernet_utils [None req-3a2f57c4-24b1-4f96-84b6-9a5333b92ced service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91894) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 19:14:46.695377 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f57c4-24b1-4f96-84b6-9a5333b92ced service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 19:14:46.696992 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f57c4-24b1-4f96-84b6-9a5333b92ced service placement] RBAC: Authorization granted {{(pid=91894) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 19:14:46.707052 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: [pid: 91894|app: 0|req: 1457/2925] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Aug 3 19:14:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 19:14:56.702755 dv-cinder-851640-2-iscsi devstack@keystone.service[91894]: DEBUG dbcounter [-] [91894] Writing DB stats keystone:SELECT=58 {{(pid=91894) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}